Unit Test

non-XP terminology (?), not quite synonymous with ProgrammerTest. A UnitTest measures a unit of software, without specifying why. A ProgrammerTest assists programmers in development, without specifying how. All runround..wondering.. What To Do Best and How ??

(Note that although most of the text on this Wiki uses the term UnitTest, the canonical ExtremeProgramming name has been changed to ProgrammerTest.)

UnitTests are programs written to run in batches and test classes. Each typically sends a class a fixed message and verifies it returns the predicted answer.

UnitTests are a key component of software engineering, and the ExtremeProgramming discipline leverages them to permit easy code changes. Developers write tests for every class they produce. The tests are intended to test every aspect of the class that could conceivably not work. (Do test adding and removing, don't test accessors.)

Key here is to CodeUnitTestFirst.

UnitTests are all combined into a huge suite of tests, using KentBeck's TestingFramework. When developers go to release new code, they run all the unit tests, not just theirs, on the integration machine. The tests must run at 100%. If any test fails, they figure out why and fix the problem. The problem certainly resides in something they did ... since they know the tests ran at 100% the last time anything was released.

Of course, sometimes the tests let something slip through. When that happens, developers unconditionally enhance the unit tests so that that problem, and any similar one that comes to mind, won't happen again.

-- RonJeffries

C3 [ChryslerComprehensiveCompensation] has over 1300 unit tests, performing over 13,000 individual checks. They run in about 10 minutes in VisualWorks. -- RonJeffries

Also, did you introduce tests late in development that identified unforeseen problems? If so, did your fixes to those problems break other things? In other words, how did you fight the "if it ain't broke don't fix it" attitude? -- KieranBarry?

It is the UnitTests themselves that allow you to get away from the "if it ain't broke don't fix it attitude." You must be able to modify anything in the entire system at anytime. Ah the freedom of it! The only way you can do that is to require the UnitTests be run before any code is released at a SingleReleasePoint. -- DonWells

What about things that can't be tested without human interaction? For example, suppose you have some code that handles printing. (And suppose the code is sufficiently deep in the system that it's not directly tested by a customer-written acceptance test. And suppose that it is necessary to test it at each integration, because breaking it can have subtle effects that will not easily be identified with this code without unit tests.) For most projects, the only practical way to test this printing code is to have a human inspect the paper that comes out of the printer and verify that it appears as it should. How does the XP testing process deal with this?

If you CodeUnitTestFirst your design will be far different from what you might expect. Most likely you will isolate the code which actually prints something to a physical printer from the code which formats something to be printed. Once the physical printer driver is tested it is unlikely to be changed anytime soon. That puts it into the not-likely-to-break category and is then exempt from unit tests. If it ever does change it is tested again by inspection of the physical output. The code that formats output destined for printing can be unit tested as usual.

Also remember that AcceptanceTests (XP style) cover anything that is of value to the customer. If it is not valuable enough to figure out how to test it then it should be removed immediately if not sooner. Where there is a will there is a way to test. -- DonWells

Creating ArtificialIntelligence algorithms like ArtificialNeuralNetwork or GeneticAlgorithm usually starts with creating a testset. This testset is often used in the process of creating the algorithm, for example for training a neural network or for the selection in genetic programming.

I just came across this paper, which oozes XP-ness: GuerillaHci on usability testing. It's by JakobNielsen, author of UsabilityEngineering, and though he doesn't mention the word UnitTest (he talks of 'scenarios'), it is a kind of unit testing for usability (of course, it's not automated; but his tests are not meant to be full acceptance tests). Interesting reading. -- BrianEwins

Has anyone tried a local Wiki to store unit tests in? -- ThaddeusOlczyk

I find it hard to write UnitTests where the unit being tested is heavily network-related, especially where it relies on components across the network to function properly. Does anyone have experience with UnitTestingNetworkFunctionality? -- GabrielWachob

I would suggest looking at MockObject to help here. Your code should be designed to separate the model from network interaction (perhaps through appropriate APIs). The model can be tested properly with a set of MockObjects implementing your networking API. Testing the networking API itself can be done without involving your model too. But then again, your particular problem might just be the exception to the rule. -- ChanningWalton

What if you have most of your code outside the classes? Like in PhpLanguage, with PhpUnit. -- DavidEscala

Maybe you are not programming object oriented? -- XaviAlbaladejo?

Yes. In php there is almost always some code outside the classes. I know I ought to make everything ObjectOriented, but sometimes it is difficult to do so. -- DavidEscala

The key point of testing, of course, is that you get good enough test coverage, cheaply enough. This is why unit tests are good: you can test the same things with functional tests, but you can generally do it more easily and cheaply with unit tests. That is why the real trick is not to work harder on writing the tests, but to write code that is more easily testable. When you take this approach, I think you'll find your logic naturally migrating out of pages and into classes.

Should unit testing test the public interface of a class or the private implementation?

(Apologies if this is an old chestnut)

I'm coding in Java and I frequently find that I want to test private methods of my classes in isolation. I usually resort to giving the methods default access instead and testing from a test case in the same package. However, this smells and feels like a violation of encapsulation.

I've seen a number of commentators state that unit tests should all be done against the PublicInterface. However, frequently when I resort to testing the public interface, my tests become harder to construct and take on the feel of acceptance tests. This is perhaps because my objects are too big. However, if I were to break them down into smaller public classes to facilitate testing, aren't these additional public classes just exposing the implementation anyway?

Perhaps I'm really missing the point given Michael Hills comment above in Are we testing too much? about testing privates? I believe unit testing is about testing method implementations not public interfaces, but there doesn't seem an easy way to do this in languages like Java. Now I'm really missing that much slighted friend declaration from my C++ days!

-- MartinBayly?

See ExtremeProgrammingTestingPrivateMethods, and MethodsShouldBePublic. You specifically do not want to test method implementations unless you are really desperate to put in some tests (perhaps when UnitTestingLegacyCode). If the implementation changes, the tests break; not fun. You don't want the tests to force you into a particular implementation. You want the tests to work as scaffolding, not a cage.

I've found that when I really get into testing, I end up refactoring the code into LotsOfShortMethods, most public or package, and most with their own tests. There are several ways to deal with this "interface pollution":

1. Ignore it
2. ReFactor a big class into many smaller classes and use delegation instead of having the class do everything on its own
3. Use package visibility rather than public
4. Expose your class through an interface that doesn't use all the methods

Recently I have been involved in lots of discussions about WritingTestableClasses. The essence of this is that often you morph your ClassUnderTest to facilitate good testing.

-- MikePorter

This is where ReFactoring ties in.

I think unit tests should test the interface: if the interface didn't change, how could it break the application?... invariants (ClassInvariant) are good for testing internals. -- PanuKalliokoski

UnitTests for XML/XSLT

I am really stuck on testing XSLT (and CSS as well). Things in my world seem to be headed toward XML for data and XSLT for presentation and transformation (with possible use of CSS, XFORM, etc.).

It seems that future applications may consist of little more than XML, XSLT and maybe a little script glue.

Anyone care to make any suggestions on how to apply XP unit testing on this stuff? Object-oriented it ain't!

-- DanVokt

See XmlUnit, XsltUnit

For data transformers (filters) such as XSLT, a good unit testing method is often to drive the transformation once, check that it is what was intended, store the result, and run diff -u against the stored result and the current result of the transformation. -- PanuKalliokoski

There is another approach. It is possible to verify transformation logic of one template only: whether it is invoked via call-template or apply-templates. Some time ago I've created a small Java library allowing to call or apply transformations of one template and then to verify result of the transformation. Verification can be achieved via XPath. I did not have any time to publish this tool, or to continue its developing, so if anyone interested in it one could contact me. -- PavelSher

Are we confusing tradition with necessity?

Perhaps the distinction between an AcceptanceTest (nee "functional test") and a UnitTest is not what they test or how they're done, but in who they satisfy. The acceptance test makes the customer satisfied that the software provides the business value that makes them willing to pay for it. The unit test makes the programmer satisfied that the software does what the programmer thinks it does.

It could be that some people confuse traditions with the actual definition of the tests. One might say, for example, "A unit test is testing an individual class in isolation," or "An acceptance test tests the entire program." That would be wrong. They are not the definitions, just traditions resulting from the forces that act upon you when you are doing testing. There is absolutely nothing wrong with the programmer writing a so-called unit test that tests the whole program, or with the customer defining an acceptance test that stubs out part of the system. It's up to the people doing it to weigh the costs and benefits, not to be arbitrarily constrained to tradition.

However, some authors such as MartinFowler and MatthewFoemmel? believe that "Unit tests are written by developers and typically test an individual class or small group of classes." [http://www.martinfowler.com/articles/continuousIntegration.html#N222] It may be that the authors are providing definitions or it may be that they are acknowledging a traditional understanding of the terms. In any case, MartinFowler would probably be the first to say do what works and not be constrained to traditions when they do not serve well.

[DoBothUnitAndAcceptanceTests.]

From the XP mailing list:

I'm trying to introduce some XP practices where I work. People are resisting unit tests for a couple of reasons and I'm looking for help. One of the problems is that the tests are fragile in the face of XP's dynamic and evolving design. Someone goes to the effort of writing a bunch of unit tests, and then they realize that their design was wrong. To change the design they have to modify some or all of the tests. Typically they just comment out the tests. The feeling is that it would be easier to write the tests once the design has solidified a bit more. -- Chris

• Is the testing framework awkward? Most of the time, it should be easier to add a new test than to add the code that passes it.

• Are we writing mini-functionals? A common newbie mis-step is to write UnitTests which focus on an object's behavior in complex interactions with other objects. Focus on isolated behavior. Sometimes tests can be mini-functionals, but we can move with more confidence if we know that the collaborators are themselves thoroughly tested.

• But we can't test this object without pulling in all those other objects. This is often a smell. Objects should be tiny and stupid. That way we can write tests that are tiny and stupid. Bang at the object until you can test it in isolation. Or abandon the design that created that object. Investigate MockObjects and other strategies that help to put one object at a time under the microscope.

• Are the tests looking too deep into the guts? If our tests poke too deep into mechanisms, then every time we change the mechanism we have to change all the tests. The less you expose to the world, the fewer the unit tests you will write.

• Are we testing too much? The purpose of unit testing is to increase developer mobility. It is not to verify the correctness of your program. Are you testing privates? Are you testing simple accessors? Are you trying to make unit tests perform 100% code coverage? These are all practices that may have some value, but also may serve to defeat the real purpose of unit tests.

It's a wonderful humbling experience, UnitTests. I don't write code any other way anymore. My code has less problems, I have more confidence and management has more confidence. -- sg

I never see anyone notice this here but it happens to be a very nice improvement in my working process. Am I just unlucky working on application with heavy setup or does other people see this as an basic GoodSense? improvement ?

I HaveThisPattern. The current behaviour I'm testing would take tedious setup and 10 to 15 minutes of running to get to without UnitTests. Not only would this be a boring pain in the ass, but it would be a constant flow breaker. Also, it's intricate enough that even with a written down test procedure, I'd probably never end up refactoring the code once it works; I'd not want to go through and manually test the code again, when it's not even fresh in my mind. Unfortunately, I'm still arranging the app to be easier to set up state to test other such situations. -- JoeWeaver

All runround..wondering.. What To Do Best and How ??

See

Specific tips on doing UnitTests:

UnitTestTechniques, XpTestFaq, UnitTestExamples, CodeUnitTestFirst, TestEverythingThatCouldPossiblyBreak, OptimizingUnitTests, UnitTestExamplesAndGuidelines, MockObject, UnitTestingErrors, UnitTestingCorba

UnitTestsAndDatabases, GuiUnitTesting, UnitTestingMarsOrbiters, UnitTestingLegacyCode, UnitTestsForSynchronizationLogic, TestPrintedOutput, UnitTestsForLibraries

DontChangeTheCodeTest, NoTestsYetTest, UntestableUnits

UnitTestingIsDesign, ArguingThroughUnitTests, ResilienceVsAnticipation, ProgrammerTest

RenamingUnitTests, UnitTestsDefined, XpVsStandardDefinitionOfUnitTest, ProgrammerTest

UnitTestingCostsBenefits, UnitTestTrial, SourceTest, QaIsNotQc, IsUnitTestingExtreme, AcceptanceTests, SeparateCodingAndTesting, PoorMansTestingFramework

This page mirrored in ExtremeProgrammingRoadmap as of April 29, 2006