Testing Resources

Terminology

Class Under Test (cut)

The class whose functionality is being tested.

Test

A single execution within a larger test suite.

Test Class/Test Fixture

A class which primarily contains test methods.

Test Suite

A collection of tests, executed as a group within the testing tool. May be defined within a single class, or from within several classes.

Dependency

A resource (network, file, or database), object instance, or class of object instances, which is needed by a CUT in order to perform its duties.

Stand-ins

An object instance which takes the place of a dependency. Generally, these can be further classified two ways:

Dummy/Stub/Fake

A stand-in object that provides only sufficient method implementations for fulfilling the requirements of the test.

Mock

A dummy/stub which also provides mechanisms for verifying which methods were called and/or verifies the arguments used when calling the method.

Types of Testing

Unit Testing

• see also: Microtest
• Exercises a single class in isolation, providing dummy/stub/mock implementations for dependencies only as needed to complete a test.
• Should not rely on external resources such as database connections, network connections, or files.
• Should be the fastest type of tests, examples of test suites with hundreds of true unit tests can be found that run in just a few seconds.
• Sometimes may be the hardest to create, as you may have to introduce mechanisms whereby you can inject alternative implementations of dependencies.

Module-level Testing

• Exercises a logical group of related classes together as a unit larger than a single class, providing dummy/stub/mock implementations for external dependencies as needed.
• Should not rely on external resources such as database connections, network connections, or files.
• Depending on the content of the tests, should run almost as fast as unit tests.
• Since these tests do not attempt to provide dummy/stub/mock implementations for all dependencies, these tests may be somewhat easier to create than unit tests.

Integration Testing

• Exercises a single module, adding in the external dependencies (database, network, or files), or exercises a group of modules together in a tightly-controlled way.
• Require a high level of integration between the classes under test, but does not require a container.
• These may be very long-running tests, but can still provide some value even if run only once a day.

System Testing

• Exercises an entire system after it is deployed to an appropriate container.
  • Running a test scenario against a deployed application, for example.
• Includes accesses to databases, network locations, and the filesystem.
• Massive amount of set-up needed compared to the finer-grained tests.
• Typically very fragile, as the external dependencies (database, network, or filesystem) may change between runs of the test.
• May only be run once or twice within an iteration, typically to provide user-acceptance feedback.

Controlling Dependencies

The first goal of creating thorough tests is to gain control of all of the dependencies in the class. There are a few ways to do this, but the most prevalent is through the use of properties in classes. That is define a private field, a public setter, and a protected getter. This combination of java permissions allows the class to access to its field, allows sub-classes to access the field, allows the class to control the assignment of that field, and allows for injection of the dependent property.

new Is Not Always The Best Way To Create Dependencies

When you create an instance of a dependency using new within the code in a class, the tests are no longer able to control or inspect the state of that dependency. There are many cases where this is okay: the dependency is only used temporarily, the dependency is a fundamental class which doesn't need to be controlled, or the dependency is so simple that there is no need to validate its behavior. For most non-trivial cases, it is better to create an injectable property, than it is to create the dependency within the production class.

But Testable Classes Are Too Open

TOUGH! Which is better: code that has been tested and could be extended/fiddled with in the wrong way or code that cannot be tested?

Effects of Project Structure on Tests

The project structure chosen by Maven seems to be the best way of structuring tests. Test classes live in a package structure identical to the production classes, but has a different location. When the code in both directories is compiled into the same location, test classes end up being in the same location as their production counterparts. This allows the test code to have access to the protected members of the production code, allowing for testing of internal interfaces of the production class.

One Production Class != One Test Class

It is not always necessary to create one test class for each production class. It is common to create a single test class for multiple closely-related classes (i.e. multiple implementations of a small interface). It is also common for there to be unit tests for a production class, module tests for the classes related to the single production class, and integration tests for the package containing the individual production class

Other Resources

Test Infected

Essay on the creation of JUnit from Kent Beck.

Working Effectively with Legacy Code

Michael Feather's book on introducing automated tests to code which currently does not support automated tests (i.e. making untestable code, testable).

Refactoring: Improving the Design of Existing Code

Martin Fowler's book on introducing changes to design without introducing change to functionality.