Share and Enjoy
David Saff's blog of technological joys.Thursday, December 07, 2006
New paper: The Practice of Theories
[Copyright notice: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.]
The Practice of Theories: Adding "For-all" Statements to "There-Exists" Tests (David Saff, Marat Boshernitsan) Submitted to IEEE Software, Special Issue on Test-Driven Development.
Abstract: Traditional unit tests in test-driven development compare a few concrete example executions against the developer’s definition of correct behavior. However, a developer knows more about how a program should behave than can be expressed through concrete examples. These general insights can be captured as theories, which precisely express software properties over potentially infinite sets of values. Combining tests with theories allows developers to say what they mean, and guarantee that their code is intuitively correct, with less effort. The consistent format of theories enables automatic tools to generate or discover values that violate these properties, discovering bugs that developers didn't think to test for.
Great, thought-provoking article which covered new ground for me and neatly summarised other tangled thoughts that I've had on this subject. Is the Theory Runner available for playing with?
Btw, I do own Mark Pilgrim's book, but DiveIntoPython might be a good citation / reference rather than the published book. Just a thought.
Thanks! The Theory Runner is not yet available for public consumption, but I'm working to make it so (as a side project).
I read Dive Into Python from the website, but (a) from my academic experience, I prefer citing printed material, (b) I thought readers might like the chance to pay Mark for his excellent book.
void hashCodeIsInSyncWithEquals(Object a, Object b) {
assumeEquals(a, b);
assertEquals(a.hashCode(), b.hashCode());
}
What if a.hashCode() equal to b.hashCode() but a isn't equal to b? How would you fix this? Write another Theory?
How would you test theories? Theories might be more complicated and error prone than other kind of tests aren't they?
I don't believe there's a bug. Giving every object the same hashcode is inefficient but meets the hashCode specification.
If a theory is incorrect because it fails when it should pass, this can be found either with user-provided data, or data generated by automatic test-input-generation tools like JUnit Factory. More difficult is if a theory passes when it should fail--this is a problem with tests, too, but the mental effort required to catch the problem is more.
<< Home
Archives
February 2005 June 2005 March 2006 August 2006 December 2006 April 2007 May 2007 January 2008
Subscribe to Posts [Atom]