Standard Definition Of Unit Test

The "standard" definition of UnitTest, used within the "professional QA testing community" is given.

Mika Moilanen <mika.moilanen@shire.ntc.nokia.com> defines UnitTest, in a message on some news group -- below the double lines below.

The definition was taken from some "testing" news group.
[What's the name of this mailing list, and where might I find it?]

[This page has been invoked as an authoritative definition -- in UnitTestsReconsidered.]


Other "standard" definitions, possibly from other communities:
Unit testing
Testing of individual hardware or software units or groups of related units
-- Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY: 1990.

Better yet, but not reproducible, is CodeComplete, chapter 25.



In article <61on1osnb0u.fsf@anvil.ntc.nokia.com>, Mika Moilanen <mika.moilanen@shire.ntc.nokia.com>
responds to this question...
"I'm not sure if I understood correctly what so you mean, could you specify an example of a case where using other modules in unit testing is a problem?"
[with...]

The answer has a number of components:

   0. A "unit" (as defined in the software testing literature) is the
      smallest piece of code software that can be tested in isolation.

a. "In isolation" means separate and apart from the application, and all other units.

b. The usual connotations associated with a unit are that

- It is a compilation unit recognized by the compiler

- It occupies a single file

- It is small (in "lines of code" sense)

- It is atomic, i.e., you normally do not think of breaking the unit into smaller pieces

c. Units, in object-oriented software, are classes, metaclasses, parameterized classes, and their corresponding instances. In programming languages that allow them, stand-alone procedures and functions are also units.

1. Unit testing (as defined in the software testing literature) requires that we test the unit in isolation. That is, we want to be able to say, to a very high degree of confidence, that any actual results obtained from the execution of test cases are purely the result of the unit under test. The introduction of other units may color our results.

2. Since we can seldom, if ever, test a unit without the presence of at least some other software, unit testing involves such things as "drivers" and "stubs."

Traditionally, a driver˛ is a piece of software that controls (drives) the unit being tested. Drivers are usually thought of as invoking, or at least containing, the unit being tested, i.e., units (being tested) are subordinate to their respective drivers.

Sometimes, units require the presence of other subordinate (to the unit) software. Traditionally, a stub˛ is a piece of software that both mimics the characteristics of, and is (hopefully much) simpler than, a necessary piece of software that is immediately subordinate to the unit being tested.

Drivers are programs or tools that allow a tester to exercise/examine in a controlling manner the unit of software being tested. A driver is usually expected to provide the following:

- a means of defining, declaring, or otherwise creating, any variables, constants, or other items needed in the testing of the unit, and a means of monitoring the states of these items,

- any input and output mechanisms needed in the testing of the unit,

- a means of controlling the unit being tested, e.g., deciding when the unit will be invoked, and what information will be supplied upon invocation,

- the generation and/or handling of any necessary interrupts, and

- the generation and/or handling of any necessary exceptions.

Stubs are program units that are stand-ins˛ for the other (more complex) program units that are directly referenced by the unit being tested. Stubs are usually expected to provide the following:

- an interface that is identical to the interface that will be provided by the actual program unit, and

- the minimum acceptable behavior expected of the actual program unit. (This can be as simple as a return statement.˛)

The goal is to keep both drivers and stubs at a minimum level of complexity. If a driver or stub becomes too complex:

- it will have to be formally tested itself, and

- the risk of the driver or stub masking, or contributing to, an error increases to an unacceptable level.

The simplest driver is indeed more complex than the simplest stub. However, drivers tend to reach a (manageable) maximum level of complexity and remain there, whereas it is not uncommon to see the complexity of (at least some) stubs come very close to the complexity of the units that they are supposed to be representing.

Lastly, there are tools (e.g., test bed generators and test harnesses) that can automate the generation of drivers and stubs.

3. Of course, we need test cases. In unit testing, test cases are a systematically generated set of black-box, white-box, and gray box test cases. Test cases are created based on the specification for the unit (black-box), the code for the unit (white-box), or both (gray-box).

Running (executing) a unit test case requires that we create the necessary conditions using the drivers and stubs. If we are using generic drivers and stubs, we will have full control, and can run whatever test cases we wish. If other units are acting as drivers and stubs, they may (and often do) prevent us from running some of the test cases we have designed.

So, the use of other units in the testing of a unit creates two sets of problems:

   1. The other units may color the results of a test case, and lower
      our level of confidence in the unit under test.

2. The other units may interfere with the straightforward execution of one or more test cases.

-- Ed

********************************************************************** Edward V. Berard | Voice: (240) 271-3777 The Object Agency, Inc. | Fax: P.O. Box 4116 | E-Mail: eb@flashcom.net Gaithersburg, Maryland 20885-4116 | Home Page: http://www.toa.com/ U.S.A. | *Consulting, Training, & Research in Web/Internet/Intranet Technology*



[Original header material:]

EdBerard? complains that the XP definition of UnitTest is different than the "industry standard" definition. Can anyone throw any light on the difference?


From the newsgroup, a "standard" unit test depends only on one unit. If it uses other units, they are replaced with simple stubs for the duration of the test. The intent is to avoid bugs in the tested unit from being hidden by bugs in the units it uses.

As I understand XP, a single unit test might exercise several units at once and the writing of extensive stubs is not required. So the XP definition is broader; it includes more tests.

[I've added a bit of discussion under UnitTestDelegator]



      a. "In isolation" means separate and apart from the application,
          and all other units.

...leading to UnitTestIsolation discussion.


So maybe the only difference is the shades of "isolation" involved. LargeScaleCppSoftwareDesign says you have "enough" isolation if your testee only calls tested objects. And if XP tests all the time...


Outside of the professional "quality assurance" testing community, the term "unit testing" covers just about everything the programmer may do (or may not do ;-) to test their code -- before "integration testing."

"(Big Bang) Integration Testing" is where you put all the programs together -- produced by different individuals or groups -- and see how long it takes before you can get anything to work.

[BigBangTesting leads to IntegrationHell.]


CategoryTesting

EditText of this page (last edited February 16, 2005) or FindPage with title or text search