Share and Enjoy
David Saff's blog of technological joys.Wednesday, June 29, 2005
Continuous Testing and Cruise Control
I'm currently the primary developer of the Continuous Testing plug-in for Eclipse. I've been asked several times to explain the difference between continuous testing and the CruiseControl tool for continuous integration. Here's how I explained it recently:
What is the difference between continuous testing and cruise control?
Cruise control is a software package that implements the process of continuous integration. In the most famous article on the subject, Martin Fowler calls continuous integration"a fully automated build and test process that allows a team to build and test their software many times a day," and discusses how the CruiseControl framework was built and used at ThoughtWorks to aid in continuous integration. The idea is that once the build and test process is automated, the best way to make sure that it's run "many times a day", and to get the benefits of doing so, is to make an automated daemon responsible for running through the process on every check-in to the repository.
At another point in the same article, Fowler states "Developers typically run some subset of the unit tests with every compile as they are writing the software. This actually speeds up the development work since the tests help to find any logic errors in the code you're working on. Then, rather than debugging, you can look at the changes since you last ran the tests. Those changes should be small and thus it's a lot easier to find the bug." The idea of continuous testing is that the best way to make sure that the unit tests are run "with every compile", and to get the benefits of doing so, is to make an automated daemon responsible for doing so.
In short, continuous integration and continuous testing are both good ideas that require either developer discipline or automated support.
CruiseControl automates continuous integration to catch errors quickly:
- for an entire team
- on a shared server
- several times a day
Our plug-in automates continuous testing to catch errors quickly:
- for a single developer
- on the developer's workstation
- several times an hour, or more
What is the difference between continuous testing and cruise control?
Cruise control is a software package that implements the process of continuous integration. In the most famous article on the subject, Martin Fowler calls continuous integration"a fully automated build and test process that allows a team to build and test their software many times a day," and discusses how the CruiseControl framework was built and used at ThoughtWorks to aid in continuous integration. The idea is that once the build and test process is automated, the best way to make sure that it's run "many times a day", and to get the benefits of doing so, is to make an automated daemon responsible for running through the process on every check-in to the repository.
At another point in the same article, Fowler states "Developers typically run some subset of the unit tests with every compile as they are writing the software. This actually speeds up the development work since the tests help to find any logic errors in the code you're working on. Then, rather than debugging, you can look at the changes since you last ran the tests. Those changes should be small and thus it's a lot easier to find the bug." The idea of continuous testing is that the best way to make sure that the unit tests are run "with every compile", and to get the benefits of doing so, is to make an automated daemon responsible for doing so.
In short, continuous integration and continuous testing are both good ideas that require either developer discipline or automated support.
CruiseControl automates continuous integration to catch errors quickly:
- for an entire team
- on a shared server
- several times a day
Our plug-in automates continuous testing to catch errors quickly:
- for a single developer
- on the developer's workstation
- several times an hour, or more
Comments:
<< Home
David,
Just so you know, someone's actually watching to see what you do here... Thanks for the bullet points - it makes yours goals very clear.
It seems to me that better testing tools on an individual's workstation are the way to introduce the methodology to a group like the one I currently work for. Server-side measures are a pain for our team because there are enough problems just finding shared hard-drive space. If the resources can be found, it still means yet another application that someone has to learn and support, which is always a big deal here.
Something extra in the IDE could go a long way in encouraging people to test more frequently. Team-members regularly would disable old tests and then never re-enable them, because they took too long - something that automatically determined which tests to run might have kept those tests up-to-date.
With out of date tests, incorporating everything into a unified test framework was impossible, because without a central server, the process would be far more painful to the one who tried to integrate than to those whose tests did not work. Maybe in some cases something simple could help prevent a poor process from snowballing like that.
Post a Comment
Just so you know, someone's actually watching to see what you do here... Thanks for the bullet points - it makes yours goals very clear.
It seems to me that better testing tools on an individual's workstation are the way to introduce the methodology to a group like the one I currently work for. Server-side measures are a pain for our team because there are enough problems just finding shared hard-drive space. If the resources can be found, it still means yet another application that someone has to learn and support, which is always a big deal here.
Something extra in the IDE could go a long way in encouraging people to test more frequently. Team-members regularly would disable old tests and then never re-enable them, because they took too long - something that automatically determined which tests to run might have kept those tests up-to-date.
With out of date tests, incorporating everything into a unified test framework was impossible, because without a central server, the process would be far more painful to the one who tried to integrate than to those whose tests did not work. Maybe in some cases something simple could help prevent a poor process from snowballing like that.
<< Home
Archives
February 2005 June 2005 March 2006 August 2006 December 2006 April 2007 May 2007 January 2008
Subscribe to Posts [Atom]