Unit Tests Defined

Failure of a UnitTest implicates only one unit. You needn't look elsewhere to learn why it failed --PhlIp

Integrated UnitTests are not:

That said, let's build a UnitTest to test a class called eyeScream.

	all:
		msdev Project1.dsp /MAKE "Project1 - Win32 Debug"
		msdev eyeScream\eyeScream.dsp /MAKE "eyeScream - Win32 Debug"

Notice I did not sweat over "exporting the MAK file from the project", or calling NMAKE or CL.EXE (or even changing the current folder). I got the same compiling environment as developers get inside the IDE, by calling the IDE itself in command-line mode.

	test: all
		msdev eyeScream\eyeScreamUT\eyeScreamUT.dsp /MAKE "eyeScreamUT - Win32 Debug"
		eyeScream\eyeScreamUT\Debug\eyeScreamUT.exe

Observe we have no reason to compile UnitTests in Release mode. The hardest bugs to find are often those that only show up in Release mode; why on earth wouldn't you want your UnitTests to catch them?

Also observe the last line runs the test.

As you add more tests you just trot them out under the target 'test'. And you provide a shell short-cut to open a console and run the tests and "pause" after them so the line "All tests passed" can stimulate your pleasure centers. Then you order the crew to stimulate their pleasure centers every half hour or so.

That's all there is to it. We did it without buying more tools, or adding risk to the project, or hiring a "test engineer" (that's AcceptanceTests), or learning new languages. We got there by doing things we already know how to do, and obeying the definition of IUT:

Hammering these simple concepts into some programming shops' brains will allow them to not hear "resource-burning overhead and complicated 4th-party tools" if you ask them with feigned innocence "do you UnitTest?"


I have today fooled around with ways to do the above a bit simpler using Visual C++ 6.0, (I don't know how to use and create makefiles under Windows). I will describe what I did here so that someone can tell me if I am missing the boat, and because it might be of use to others.

I have added another project called Tests to a Demo project so that there now are two projects in the Class View. Tests are the tests that need to be done on the classes in the Demo Project. A failure of a test causes Tests to write an error to a console (printf or cout). And the Tests executable returns a 1. You will see where this goes shortly.

I have made the Demo project Dependent on the Tests project and I have added the executable of the Test project as a pre-link step in the project settings of the Demo Project.

The result of all this is that all the output of the Tests executable is printed in the compile window and that the Demo project won't link if the Tests do not work.

I think this will basically be as functional as the process with the makefiles.

If someone wants to see the VC workspace, please leave a note on my WikiPage and we can make a plan.

After some hacking I got CppUnit integrated into the VC++ 6.0 IDE as well by using the above method. It works great and I am definitely converted to UnitTesting. I am still working on the powers that be to start implementing the rest of XP.

-- LourensCoetzer


Projects fully involved with UnitTests should work from a completed framework such as TestingFramework, CppUnit, JavaUnit, etc. For Cpp, you can also use the test unit that's built into the BoostLibraries. And if you don't have Boost, you ought to get it anyway.

UnitTests permit fast & safe RefactorMercilessly sessions, and are a natural complement to DesignByContract.


Do UnitTests have to be deterministic? (I.e. You set up a fixture, provoke some canned behaviour and then test the result. Must the behaviour itself be deterministic?) --

Speaking as one who has had to UnitTest mail systems, you'll really hate yourself if the test only works most of the time. I kept coming up with excuses not to run those particular tests, because they were a) incredibly slow and b) failed about 1/6th the time because of subtle timing weirdness with the mail daemons. -- GrahamHughes

Their results have to be "testable", not "deterministic". If a function produces a pseudo-random number, you can do a few quick tests on a sample of results to see that it has a linear spread, a given range, etc. -- PhlIp

The results have to be reliable. - If they say pass, they mean pass, and if they say fail, they mean fail. When testing a unit that interfaces with unreliable or unpredictable subsystems, you have to fake those subsystems to give predictability to your UnitTest. -- WayneConrad

To discuss UnitTest isolation factors, see StandardDefinitionOfUnitTest.

To discuss unreliable subsystems, see ExtremeProgrammingChallengeNineteen.


I realize this is probably my naivete with the msdev tool, but I'm confused how to do this one thing: "provide a shell short-cut to open a console and run the tests." What is a "shell short-cut"? I assume that it's some way to invoke nmake via batch file?

I believe it means creating a short-cut on your desktop that invokes said make command. Thus, just double-clicking on the short-cut launches the tests.

Install IE4 before you go to IE5, or get Win98 or Win2000, and you get the Quick Launch bar, typically between the Start button and the Task Bar. Drop the icon there, and it's generally always available.

If you're sufficiently clever, you can use the Post Build Commands to do this for you. Then, every time you compile you will test.


So now, the question is how do structure the folder structure of the tests. I don't want to interminge code and tests because that distracts me from my code. I also don't want to hide my tests because that slows down the process.

See UnitTestFolderStructure.


Does this page yet contain a definition? There's a list of things "Integrated UnitTests are not", then a very platform-dependent example, and a lot of interesting discussion, but nothing else that looks like a definition.

Is the not-list supposed to be a sufficient definition?

-- DavidVincent

Listen, at a new job, to the following whining...

"We can't do UnitTests. We don't have...

The "definition" is the minimum needed to JUST START WRITING THE TESTS. No framework, gurus, 4th-party tools, light methodology, or excess overhead.

I recently read about PairProgrammingCostsBenefits, and this discussion makes me yearn to hear about the UnitTestingCostsBenefits. -- RobHarwood

More than a year on, I regret asking about the definition. At the time, the "discussion" above was helpful in that it spurred me to learn more about UnitTesting, XP and Wiki. Definitions are available at UnitTests and StandardDefinitionOfUnitTest. I still don't think this page's name reflects its contents very well. -- DavidVincent


See StandardDefinitionOfUnitTest, UnitTestCookbooks


CategoryTesting

EditText of this page (last edited April 10, 2012) or FindPage with title or text search