Test Plan Checklist

Is a Test Plan Necessary

Here are some areas to consider:

If you answered "Yes" to any of the above questions, it appears likely that you should do laboratory tests. If you still believe that these tests are not necessary, be sure to reference in your explanation any item marked "Yes" above. Point out that DESPITE that feature, you believe that lab tests are unnecessary, and explain why.

If you checked off "Yes" to any of the above items, and /or feel that testing your project equipment or software or configuration in a test lab is a good idea, continue to fill out this form.

Designing Test Lab/Training Environment

You can follow these steps to determine what should be tested. You may be asked to present some or all of this information at the Architecture Review.

A: List what will be tested:

1. Define the Test Points. Based on the Architecture Design, create a numbered list of all (a) pieces of hardware, (b) software applications or support systems that will be added to the current network architecture. Then list (c) all interfaces (software to software, hardware to software, hardware to hardware) between those new components listed in (a) and (b). Lastly, list (d) all interfaces between those new components and the current architecture.

2: Define Feasible Test Points: Based on your understanding of the limitations of the Test Lab, list all items from step A that you can set up in the lab (e.g., "All interfaces can be simulated.", or "All interfaces but items 3, 17, and 24 can be listed.")

3: Defend Feasibility: For each item listed in Step 2,

(a) Explain why items from Step 1 have been excluded from the list in Step 2. (e.g. "There are no funds to purchase a Model xxx PBX for the lab, and all available ones are in production environment.")

(b) Explain what the impact of a failure at that component or interface would be to the proposed Network Improvement architecture. Estimate the likelihood of that failure (e.g. "Downtime is estimated at 3 hours per year".) and explain how that estimate was generated (e.g., "The product literature uses that reliability statistic.")

B: Show the Lab Layout Plan:

Draw a diagram (e.g. in Visio) of the Test Lab environment. Note that the "Test Lab" can be at any location, even on a network, AS LONG AS IT IS IN NO WAY CONNECTED TO THE PRODUCTION IS NETWORK. It is advisable to review this layout plan with other IS Networking personnel before actually building the Lab. If while building the Lab you find you need to make changes to the Plan, edit the above Test Plan document sections for accuracy and completeness so that it reflects the actual test environment.

Determine What Tests to Run

You can follow these steps to determine what should be tested. You may be asked to present some or all of this information at the Architecture Review.

A: Inputs and outputs:

(1) For each feasible test point (listed above), define the relevant inputs and outputs. (e.g. for an interface between a new management database and a router, define the router information collected by the database, and any data collection commands from database to router. For the interface between the new router and the network router, define the traffic carried from one router to another.) List the inputs/outputs here.

B: I/O Variables:

(1) For each of the feasible test point inputs and outputs, read through the list of SLA features from the Architecture Design, and select any variable parameters that are relevant to the I/O stream. (e.g. for a router to router link, capacity and performance parameters are relevant; for most test points, availability/reliability is an important parameter, requiring long term stability testing). List the relevant parameters here.

(2) For these variable parameters, list the testable range which should cover all reasonable values (e.g. for the router to router link, throughput of 0 to 20 Mbps may be reasonable values for some projects). List the range of parameter values here.

(3) For input / output points that are similar to the current network architecture, list the baseline performance parameters (if known).

C: Configurable Variables:

(1) Hardware: Some equipment has configurable physical configurations (e.g. different port cards, IO port switch settings, arrangement of items on the cable, etc.). List or describe these configuration areas. (Note: When the optimal configuration is determined for that set of hardware / software, it should be listed under "Implications for the Deployment Plan.")

(2) Software: List the configurable menus and script files that require setting, writing, or optimizing for this application. (Note: When the optimal configuration is determined for that set of hardware / software, it should be listed under "Implications for the Deployment Plan.")

(3) Vendor Service: If any vendor service is used for transport or management, list the configurable settings requested of and provided by the vendor.

D: Monitoring / Management Interface:

Configure the monitoring / management terminal as required by the new hardware / software. Test range of possible screens / menus / commands. If it operates exactly as described by the vendor literature, acknowledge that here. If its operation differs from that described by the vendor, list the ways in which it differs here.

E: Security Interfaces

Configure read/write and read-only access to the management functions. If there are user login/password processes, use them. If it operates exactly as described by the vendor literature, acknowledge that here. If its operation differs from that described by the vendor, list the ways in which it differs here.

F: Test Goals

Define the results criteria for the testing. (e.g. architecture is functional, provides long term stability, and continues to function under heavy capacity; users find quality as good or better than before).

G: Tests to Run

Based upon the test goals, and the architectural components and interfaces, I/O values and configurable parameters, list the tests that you will run. (E.g., (1) simulate the range of IP traffic types and speeds through the router port, note actual throughput; (2) configure router port to support new feature, test latency with new feature on over range of traffic loads; etc.)

Write Test Results

A: Implications for Deployment:

When the optimal configuration is determined for hardware configuration, configurable menus and script files, or vendor service configurations, list or attach them here.

B: Implications for Support:

Support staff need to know not only the optimal configurations, but the less optimal configurations if they achieve a desirable state.

If the test has shown that variations of the optimal configuration can achieve certain localized and useful network characteristics, list and describe the configuration(s) and it's impact here. (E.g., Configuration #4 results in overall greater packet delay of 5-8%, but decreases packet delay variability, or jitter, by 70%. We chose to use configuration #3 for minimal packet delay; but if jitter becomes a problem, configuration #4 would be an acceptable solution.)

C: Training Acomplished:

(1) Note here if the people who will be deploying the network components were offered access to the test environment and a guided hands-on run-through of the steps required to physically deploy and configure them.

(2) Note here if the people who will be monitoring and managing the new network components were offered access to the test environment and a guided hands-on run-through of the steps required to monitor and maintain the new network components.

D: Other

What other information should be recorded based on the test results?

8. Responsibility:

This Test Plan will be posted by the Network Improvement manager at http://xxxx/NetImprove/Projects/Project_Name/Test_Plan. It will be the responsibility of the project leader to update this Test Plan should any changes be made to the actual test architecture or any results of new tests are collected.