Testing process overview

Table of Contents:

introduction
The simple case
The details

introduction:

SandBoss provides an integrated message-level testing infrastructure which can be leveraged for functionality verification, regression, "smoke check" and other purposes. This document provides an overview of the mechanisms involved.

Tests most commonly originate from:

verifying a node produces the correct output in response to specific input

verifying a specific system scenario works

verifying a bug no longer occurs

The goal is to make the creation of tests easy, and to have them run by default.

The simple case:

Assuming an application or deployment called "MyProj", this is how testing works in the simplest case:

The SandBoss developer uses the test script editor to define a new test script, saving it as MyProj/test/test_1.xml

The SandBoss developer uses the configuration editor to define a new configuration with the nodes required for the test. The developer saves the test as MyProj/test/config_1.xml after verifying test_1.xml is listed in the testScripts parameter of the MessageDriver node instance.

At runtime, SandBossRoot finds the test directory off of the triggering build area and loads config_1.xml, which in turn triggers the MessageDriverNode to load test_1.xml. At the conclusion of the test run, MessageTrigger signals SandRoot with success or failure.

That's about it. Everything else builds off of this, or involves execution details.

The details:

Assuming you are now familiar with the basics, here's how some of the details and additional functionality fits in:

For any given node definition source package (for example "MyNode"), the default build copies config*.xml and test*.xml from the MyNode src directory to MyProj/test/MyNode. This allows you to keep your local module testing in the same directory as your source.

If MyProj requires MyBasicApp, then the build will by default copy MyBasicApp/test into MyProj/test/MyBasicApp.

When loading configurations, SandBossRoot goes through all subdirectories first, on the idea that these represent module tests and aggregated application tests which are more likely to pinpoint problems. Within a directory, SandBossRoot goes through configuration files in alphabetical order. The overall result is a depth-first alphabetical traversal of the defined test configurations.

If a deployment has simply copied over the test directory along with the other required directories, running the system will result in a full regression suite run. By defining the last test configuration in the deployment to unconditionally signal a system shutdown, you can run a full regression suite along with your automated build.

A developer working on a specific node may wish to temporarily alter their local build file so that only their current test of interest will be run. By changing the filesets to only list the test of interest, they can focus on the problem at hand. Once their test is working they simply revert their copy of the build to the original.

A deployment may wish to test basic functionality on system startup. By defining a specific set of configurations and tests to be copied for the deployment, a "smoke check" test is set up. Unless halted, SandBossRoot will load the main configuration and start it as the last step in the system startup process.

The order in which configurations are loaded does not typically matter, since each configuration is completely self-contained. In cases where a different ordering is desired, configurations can be renamed or moved as part of the build process.

Test suite management can get as advanced as necessary through specific build targets appropriate for the deployment. For additional details, refer to the build files or the SandBossRoot package documentation.

© 2002 SAND Services Inc.
All Rights Reserved.