Test Collector

When using a unit testing framework like JavaUnit, forget adding each TestCase to the TestSuite. It's tedious and error-prone.

Just have some code loop over the directory with your TestCases, and pattern match for your tests (i.e, if your convention is to have "Foo.java" and "FooTest.java", match everything named "*Test.java").

Bonus points go to implementations that put the TestCases with the most recent modification times at the top of the list (that way, your most likely failing tests come faster, it's more feedbacky that way).

Implementations: Implementations built-in to the *Unit:

If you collect tests automatically using a TestCollector (and even if you don't), it's also a good idea to use a TestInventory to detect if tests have silently disappeared.

Is a TestCollector necessary if you NeverWriteaLineOfCodeWithoutaFailingTest? You should be expecting a RedBar when you run your test for the first time, and if it's not, you'll realise you forgot to register the new suite.

See ThreeStrikesAndYouAutomate. Why should I keep adding test suites when the computer is perfectly capable of doing it for me?
I've been thinking about writing a Java test suite collector for a while. I recently put my first attempt at a build collector on SourceForge. It builds three different test suites- AllCollectedFunctionalTests?, AllCollectedAcceptanceTests?, and AllCollectedUnitTests?.

Test collectors will work because your codebase doesn't include tests that depend on the order that they're run. Only a complete tool would commit something like that into a repository.

In ruby, I just require all the *.rb files under a tree, and then I go through all of the objects looking for ones marked appropriately. (In my case, the convention is to test Foo with a TestCase named TC_Foo.) I used actually just to run everything that inherited from TestCase, until I started creating abstract TestCases descending from TestCase (DBTestCase, WebTestCase, etc.).

	suite = Test::Unit::TestSuite.new("ruby unit tests")
	test_regex = Regexp.compile('^(.*::)?TC_[^:]+$', Regexp::EXTENDED)
	ObjectSpace.each_object(Class) { | klass |
		suite << klass.suite if test_regex.match(klass.to_s)

Requiring all the *.rb files (or some other) is a test pattern that seems quite prevalent. Maybe I should call it something like TestAllClassesTogether?. CppUnit and CppUnitLite do something like this, by automatically registering all tests with the testrunner.

However, in languages like C++ this does not test that the classes (and their associated files) can be used in isolation.

Conjecture: SmalltalkLanguage and its friends have no concept of file structure. All is workspace. Now, I know that I personally have had trouble moving stuff from one workspace to another - making sure I have all the dependencies.


View edit of April 10, 2012 or FindPage with title or text search