Skip to main content

Testing Resources

Terminology

Class Under Test (cut)
The class whose functionality is being tested.
Test
A single execution within a larger test suite.
Test Class/Test Fixture
A class which primarily contains test methods.
Test Suite
A collection of tests, executed as a group within the testing tool. May be defined within a single class, or from within several classes.
Dependency
A resource (network, file, or database), object instance, or class of object instances, which is needed by a CUT in order to perform its duties.
Stand-ins
An object instance which takes the place of a dependency. Generally, these can be further classified two ways:
Dummy/Stub/Fake
A stand-in object that provides only sufficient method implementations for fulfilling the requirements of the test.
Mock
A dummy/stub which also provides mechanisms for verifying which methods were called and/or verifies the arguments used when calling the method.

Types of Testing

Unit Testing
  • see also: Microtest
  • Exercises a single class in isolation, providing dummy/stub/mock implementations for dependencies only as needed to complete a test.
  • Should not rely on external resources such as database connections, network connections, or files.
  • Should be the fastest type of tests, examples of test suites with hundreds of true unit tests can be found that run in just a few seconds.
  • Sometimes may be the hardest to create, as you may have to introduce mechanisms whereby you can inject alternative implementations of dependencies.
Module-level Testing
  • Exercises a logical group of related classes together as a unit larger than a single class, providing dummy/stub/mock implementations for external dependencies as needed.
  • Should not rely on external resources such as database connections, network connections, or files.
  • Depending on the content of the tests, should run almost as fast as unit tests.
  • Since these tests do not attempt to provide dummy/stub/mock implementations for all dependencies, these tests may be somewhat easier to create than unit tests.
Integration Testing
  • Exercises a single module, adding in the external dependencies (database, network, or files), or exercises a group of modules together in a tightly-controlled way.
  • Require a high level of integration between the classes under test, but does not require a container.
  • These may be very long-running tests, but can still provide some value even if run only once a day.
System Testing
  • Exercises an entire system after it is deployed to an appropriate container.
    • Running a test scenario against a deployed application, for example.
  • Includes accesses to databases, network locations, and the filesystem.
  • Massive amount of set-up needed compared to the finer-grained tests.
  • Typically very fragile, as the external dependencies (database, network, or filesystem) may change between runs of the test.
  • May only be run once or twice within an iteration, typically to provide user-acceptance feedback.

Controlling Dependencies

The first goal of creating thorough tests is to gain control of all of the dependencies in the class. There are a few ways to do this, but the most prevalent is through the use of properties in classes. That is define a private field, a public setter, and a protected getter. This combination of java permissions allows the class to access to its field, allows sub-classes to access the field, allows the class to control the assignment of that field, and allows for injection of the dependent property.

new Is Not Always The Best Way To Create Dependencies

When you create an instance of a dependency using new within the code in a class, the tests are no longer able to control or inspect the state of that dependency. There are many cases where this is okay: the dependency is only used temporarily, the dependency is a fundamental class which doesn't need to be controlled, or the dependency is so simple that there is no need to validate its behavior. For most non-trivial cases, it is better to create an injectable property, than it is to create the dependency within the production class.

But Testable Classes Are Too Open

TOUGH! Which is better: code that has been tested and could be extended/fiddled with in the wrong way or code that cannot be tested?

Effects of Project Structure on Tests

The project structure chosen by Maven seems to be the best way of structuring tests. Test classes live in a package structure identical to the production classes, but has a different location. When the code in both directories is compiled into the same location, test classes end up being in the same location as their production counterparts. This allows the test code to have access to the protected members of the production code, allowing for testing of internal interfaces of the production class.

One Production Class != One Test Class

It is not always necessary to create one test class for each production class. It is common to create a single test class for multiple closely-related classes (i.e. multiple implementations of a small interface). It is also common for there to be unit tests for a production class, module tests for the classes related to the single production class, and integration tests for the package containing the individual production class

Other Resources

Test Infected
Essay on the creation of JUnit from Kent Beck.
Working Effectively with Legacy Code
Michael Feather's book on introducing automated tests to code which currently does not support automated tests (i.e. making untestable code, testable).
Refactoring: Improving the Design of Existing Code
Martin Fowler's book on introducing changes to design without introducing change to functionality.

Popular posts from this blog

Using MonoDevelop to Create an ASP.NET Web Service

NOTE: instructions below are for MonoDevelop 2.6 Beta 2 - built on 2011-04-06 03:37:58+0000Getting StartedCreate a new ASP.NET Web Application in MonoDevelop: From the menu, select: File → New → Solution…Expand C#.Select ASP.NET → Web Application.Enter a name for the ASP.NET project that will be created in the solution in Name:.Change the root location for the solution in Location:, if desired.Change the name of the root solution in Solution Name:, if desired.The Results – IWhat you have after executing the new ASP.NET Web Application project wizard is a solution containing one ASP.NET Web Application project. In the default project view in MonoDevelop, you'll find the following items: Default.aspx – This is the default web form rendered and presented in the browser when http://<server>:<port>/ is accessed. Default.aspx.cs – This C# file contains the developer-created common code and event handlers which can be used to affect the processing of the form.Default.aspx.des…

Testing Toolbelt: SpringJUnit4ClassRunner

The org.springframework.test.context.junit4.SpringJUnit4ClassRunner class is another implementation of the JUnit TestRunner class which is used to enable various features of Spring for every run of the test class and every test within it. To use the features provided by the SpringJUnit4ClassRunner class, you need to mark the class using the RunWith annotation using SpringJUnit4ClassRunner as its parameter.In addition to the custom test runner, you will want to mark the class with the ContextConfiguration annotation. The ContextConfiguration annotation is used to mark classes which will automatically read a Spring configuration file and use it to create an ApplicationContext. By default, this file located at <package path>/<test class name>-context.xml. Use the locations argument to over-ride.The ApplicationContext used by the Spring-integrated test will only be loaded once for the whole test class. This behavior can be over-ridden by annotating a test method with the Dirti…