After the announcement of our testing service I asked for feedback from prominent community members. The feedback was overwhelmingly positive with some notable quotes being:
Usability testing of Drupal 7 would have been virtually impossible if it weren't for the automated testing in place keeping core stable.
Drupal 7 HEAD is much more stable than any release we [have] ever had.
It easily saved me a 100 hours knowing I didn't break something I wrote an hour ago. I'm not sure I would have been able to have completed it [DBNTG].
Testing.drupal.org went live in October 2008, and once again, Drupal's development process was reveolutionized. Now, developers don't need to sit through a test run (which could take 30 minutes or more) in order to verify their changes are working; they can simply upload their changes and be informed by one of the testing clients.
In order to allow Drupal shops to gain the same advantages that Drupal core has received we offer our testing service. Through the service we will maintain an automated testing network similar to testing.drupal.org for use by our clients. In addition we can review your tests to ensure that they are up to standards and provide feedback on ways to improve them.
We are currently working to provide testing of "data sets" which is extremely useful for testing changes against deployed sites. The data from a deployed site can be backed up and tests run against that dataset to ensure that code changes do not break custom workflows or configurations. Testing of data sets does not replace functional testing like that used in Drupal 7 core, but it provides an additional layer of confidence when deploying changes to a live site.
Boombatower Development is adding automated testing as a new service. Boombatower Test Services (BTS) will provide a manageable solution that requires minimal setup.
One of the most exciting features included in Drupal 7 is the testing framework and the large number of tests that come with it. The testing framework has introduced a new development paradigm into core development that has opened up a number of exciting possibilities in addition to maintaining a very stable development version of Drupal core. As respected core developer Károly "chx" Négyesi said:
"Drupal 7 HEAD is much more stable than any release we [have] ever had."
The increased stability has allowed for the Drupal 7 development time to be extended and the related code freeze to be shortened. Drupal shops can benefit from the same principle. By having extensive tests in place companies can maintain a higher level of confidence in their products and extend that confidence to their clients.
BTS will use the second generation framework soon to be in place on drupal.org and testing.drupal.org. The framework has proven to be robust and the latest version will provide a number of powerful new features.
In addition to you writing functional tests for your custom modules like those included in Drupal 7 core, Boombatower will be providing a solution that allows for testing with a copy of live data to ensure that code changes do not cause issues with a fully configured site setup. Having both types of tests in place that are automatically run on all patches, commits, or nightly builds will give Drupal shops more confidence when deploying new features or code changes to a live site.
In addition to providing automated testing we also offer consulting services for evaluating tests. Properly written tests will ensure that the results received from automated testing are as useful as possible.
I will be speaking at Drupalcon DC 2009 about the history of the automated testing framework, its present form and the exciting new future. I will stick around afterwards to discuss the session and the launch of our automated testing service. If you are interested please stop by after the session.