Feedback-Directed Random Test Generation
- Carlos Pacheco ,
- Shuvendu Lahiri ,
- Michael D. Ernst ,
- Thomas Ball
International Conference on Software Engineering (ICSE '07) |
Published by IEEE Computer Society
We present a technique that improves random test generation by incorporating feedback obtained from executing test inputs as they are created. Our technique builds inputs incrementally by randomly selecting a method call to apply and finding arguments from among previously-constructed inputs. As soon as an input is built, it is executed and checked against a set of contracts and filters. The result of the execution determines whether the input is redundant, illegal, contract-violating, or useful for generating more inputs. The technique outputs a test suite consisting of unit tests for the classes under test. Passing tests can be used to ensure that code contracts are preserved across program changes; failing tests (that violate one or more contract) point to potential errors that should be corrected. When applied to 14 widely-used libraries comprising 780KLOC, feedback-directed random test generation finds many serious, previously-unknown errors. Compared with both systematic test generation and undirected random test generation, feedback-directed random test generation finds more errors, finds more severe errors, and produces fewer redundant tests.
Copyright © 2007 IEEE. Reprinted from IEEE Computer Society.This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org.By choosing to view this document, you agree to all provisions of the copyright laws protecting it.