Project

Back to overview

AVATAR: Automated coVerAge-based Testing via static-dynamic Abstraction Refinement and coarsening

English title AVATAR: Automated coVerAge-based Testing via static-dynamic Abstraction Refinement and coarsening
Applicant Pezzè Mauro
Number 132666
Funding scheme Project funding (Div. I-III)
Research institution Istituto del Software (SI) Facoltà di scienze informatiche
Institution of higher education Università della Svizzera italiana - USI
Main discipline Information Technology
Start/End 01.10.2010 - 31.08.2013
Approved amount 200'152.00
Show all

Keywords (12)

software testing and analysis; structural testing; combination of static and dynamic program analysis; concolic execution; abstract interpretation; data flow testing; Software testing; structural coverage; test case generation; dynamic analysis; static analysis; abstraction refinement and coarsening

Lay Summary (English)

Lead
Lay summary
Structural testing coverage is a proxy measure of the adequacy of testing that can be useful to both software quality managers and tester designers. Despite this, few of the many structural coverage criteria studied in academia find common application in industrial practice, due to both the scarce availability of automated support for generating test suites with high coverage and the penalizing impact of infeasible program elements.This project aims to define a new generation of structural testing approaches to automatically generate test suites with high structural coverage, and correctly account for a relevant number of infeasible elements of the coverage domain.This project will suitably exploit the combination of dynamic and static analysis techniques. Static program analyses quickly explode in complexity and become intractable, while dynamic program analyses scale to large programs, but do not cover the state space exhaustively. Recently static and dynamic analyses have been combined to exploits the positive synergy of observed facts (brought by dynamic analysis) and deductions that can be generalized from these (brought by static analysis).This project will integrate data from concrete executions of the program with the statically analysis of an abstract (over-approximated) and finite model of the program state space. It will study the use of abstraction refinement and coarsening (ARC) that we have introduced in a recent paper, to re-aggregate the states generated by the refinement process as the analysis of the program progresses. Our preliminary results based on combining abstract refinement with concolic execution to refine branch coverage are very encouraging. This project will exploit these results by exploring the applicability of coarsening to the combination of other static and dynamic analysis techniques to refine complex coverage criteria.The detailed goals of the project are to (1) define new approaches to structural testing to achieve accurate coverage according to different classes of coverage criteria, (2) define new program analysis algorithms that combine static and dynamic analysis to efficiently generate test suites and analyze infeasible targets throughout industrial-size programs, (3) define new reference abstractions that allow for optimizing the trade-off between precision and costs of the analysis for different classes of structural testing criteria, and (4) study the effectiveness of different coverage criteria by evaluating to what extent the test suites that satisfy the coverage criteria can expose program defects, and by comparing the effectiveness across different classes of coverage criteria.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
Quantifying the Complexity of Dataflow Testing
Denaro Giovanni, Pezzè Mauro, Vivanti Mattia (2013), Quantifying the Complexity of Dataflow Testing, in 8th International Workshop on Automation of Software Testing, San Francisco (USA)IEEE Press, Pisataway.
An industrial case study of the effectiveness of test generators
Braione P., Denaro G., Mattavelli A., Vivanti M., Muhammad A. (2012), An industrial case study of the effectiveness of test generators, in Proceedings of the 7th International Workshop on Automation of Software Testing (AST 2012), ACM Press, NY.
Supporting Test Suite Evolution through Test Case Adaptation
MirzaAghaei Mehdi, Pastore Fabrizio, Pezzé Mauro (2012), Supporting Test Suite Evolution through Test Case Adaptation, in Proceeding of the 5th International Conference on Software Testing, Verification and Validation, IEEE Press, Piscataway.
Automatic structural testing with abstraction refinement and coarsening
Baluda Mauro (2011), Automatic structural testing with abstraction refinement and coarsening, in 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of Sofware Engineering, Szeged, HungaryACM, Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of sof.
Dynamic Analysis for Diagnosing Integration Faults
Mariani Leonardo, Pastore Fabrizio, Pezzé Mauro (2011), Dynamic Analysis for Diagnosing Integration Faults, in IEEE Transactions on Software Engineering, 37(4), 486-508.
Enhancing structural software coverage by incrementally computing branch executability
BaludaMauro, Braione Pietro, Denaro Giovanni, Pezzè Mauro (2011), Enhancing structural software coverage by incrementally computing branch executability, in Software Quality Journal, 19(4), 725-751.
Structural Coverage of Feasible Code
Baluda Mauro, Braione Pietro, Denaro Giovanni, Pezzè Mauro (2010), Structural Coverage of Feasible Code, in IEEE (ed.), IEEE Press, New York, 59-66.

Collaboration

Group / person Country
Types of collaboration
VTT Technical Research Centre of Finland Finland (Europe)
- Publication
- Industry/business/other use-inspired collaboration
Università di Milano Bicocca Italy (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Conference Individual talk International Conference on Software Engineering (CSE) 20.05.2013 San Francisco, United States of America Pezzè Mauro; Vivanti Mattia; Baluda Mauro;
conference Individual talk ICSM'12: International conference on Software Maintenance 23.09.2012 Riva del Garda, Italy, Italy Pezzè Mauro;
Conference Individual talk Balcan Conference in Informatics 16.09.2012 Novi Sad, Serbia, Serbien Pezzè Mauro;
Conference Individual talk ISSTA'12: ACM International Conference on Software Testing and Analysis 15.07.2012 Minneapolis, United States of America, United States of America Pezzè Mauro;
research internship Talk given at a conference Summer research internship in the Mobile Enterprise Software group at IBM T. J. Watson Research Center in Hawthorne 11.06.2012 Watson Research Center, United States of America Baluda Mauro;
Conference Talk given at a conference Student Volunteer at 34th International Conference on Software Engineering 04.06.2012 Zurich, Switzerland, Switzerland Baluda Mauro; Vivanti Mattia;
Workshop Individual talk The 20th CREST Open Workshop: The Oracle Problem for Automated Software Testing 21.05.2012 London, UK, Great Britain and Northern Ireland Pezzè Mauro;
Workshop Individual talk Mysore Park Workshop on The Future of Debugging 28.02.2012 Mysore, India, India Pezzè Mauro;
summer school Talk given at a conference INRIA summer school on modelling and verifying algorithms 10.11.2011 Paris, France, France Baluda Mauro;
Conference Individual talk ESEC/FSE'11: ACM SIGSOFT Symposium on the Foundations of Software Engineering 01.09.2011 Szeged, Hungary, Hungary Pezzè Mauro;
Conference Individual talk ICSE'11: 33th International Conference on Software Engineering 21.05.2011 Honolulu, United States of America, United States of America Pezzè Mauro;
conference Individual talk GT-VMT'11: International Workshop on Graph Transformation and Visual Modeling Techniques 02.04.2011 Saarbrücken, Germany, Germany Pezzè Mauro;
conference Talk given at a conference ICST'11: 4th IEEE International Conference on Software Testing, Verification and Validation 21.03.2011 Berlin, Germany, Germany Pezzè Mauro;
seminar Individual talk Dagstuhl seminar on Self-Repairing software 06.02.2011 Wadern-Dagstuhl, Germany, Germany Pezzè Mauro;
Conference Individual talk Software Quality Days 18.01.2011 Vienna, Austria, Austria Pezzè Mauro;


Associated projects

Number Title Start Funding scheme
146831 DyStaCCo: Dynamic and STatic Analysis Combined for code COverage 01.09.2013 Project funding (Div. I-III)

Abstract

Structural testing coverage is a proxy measure of the adequacy of testing that can be useful to both software quality managers and tester designers. Despite this, few of the many structural coverage criteria studied in academia find common application in industrial practice, due to both the scarce availability of automated support for generating test suites with high coverage and the penalizing impact of infeasible program elements. This project aims to define a new generation of structural testing approaches to automatically generate test suites with high structural coverage, and correctly account for a relevant number of infeasible elements of the coverage domain. The results of the project will enable test designers to automatically compute accurate coverage measurements, thus breaking through the barriers that challenge the applicability of structural coverage criteria, and equipping the scientific community with new means for gathering credible and conclusive evidence on the effectiveness of structural coverage criteria.This project will suitably exploit the combination of dynamic and static analysis techniques. Static program analyses quickly explode in complexity and become intractable, while dynamic program analyses scale to large programs, but do not cover the state space exhaustively. Recently static and dynamic analyses have been combined to exploits the positive synergy of observed facts (brought by dynamic analysis) and deductions that can be generalized from these (brought by static analysis). This project will integrate data from concrete executions of the program with the statically analysis of an abstract (over-approximated) and finite model of the program state space. It will study the use of abstraction refinement and coarsening (ARC) that we have introduced in a recent paper, to re-aggregate the states generated by the refinement process as the analysis of the program progresses. The rationale behind ARC elaborates on the observation that the analysis algorithms generate a lot of information (refinements) to reason on the reachability of the target program elements; while the analysis meets such goal for any target elements, coarsening drops the refinements generated throughout the finalized decision processes.Our preliminary results based on combining abstract refinement with concolic execution to refine branch coverage are very encouraging. This project will exploit these results by exploring the applicability of coarsening to the combination of other static and dynamic analysis techniques to refine complex coverage criteria.The detailed goals of the project are to (1) define new approaches to structural testing to achieve accurate coverage according to different classes of coverage criteria, (2) define new program analysis algorithms that combine static and dynamic analysis to efficiently generate test suites and analyze infeasible targets throughout industrial-size programs, (3) define new reference abstractions that allow for optimizing the trade-off between precision and costs of the analysis for different classes of structural testing criteria, and (4) study the effectiveness of different coverage criteria by evaluating to what extent the test suites that satisfy the coverage criteria can expose program defects, and by comparing the effectiveness across different classes of coverage criteria. The project combines theoretical investigation with experimental validation through prototype implementations that will be defined during the project.
-