Testing is Rocket Science Not Brain Surgery

“Planning involves projecting our thoughts forward in time and space to influence events before they occur rather than merely responding to events as they occur. This means contemplating and evaluating potential decisions and actions in advance.”
—Marine Corps Doctrinal Publication (MCDP) 5, Planning

With the trend to standardize testing practices, most organizations now require a Test Strategy and a Test Plan (name, of course, dependant on the organization). Most organizations use a template to help write those documents, and all too often, the result is often a document nobody fully reads, nobody remembers, and often, that testers find themselves in conflict with as they actually start executing.

With that baseline, we have plenty of room for improvement.

I refer to the MCDP5 , for several reasons:
· The Marine Corps, as an organization, has a great deal of interest and experience in growing leadership, management, and planning capabilities.
· The Marine Corps has a long institutional history to draw experience from.
· The document is a fantastic read for anyone in management or interested in management, and many of its facets translate reasonably well to business planning.

MCDP 5 defines the key functions of planning as:
· Direct and coordinate actions.
· Develop a shared situational awareness.
· Generate expectations about how actions will evolve and how they will affect the desired outcome.
· Support the exercise of initiative.
· Shape the thinking of planners.

Since different organizations use the Test Strategy and Test Plan differently, I’ll refer to them henceforth as simply the Test Plan. To design a test plan that meets the MCDP 5 definition, we’ll take each function, and map it to items we can use to build our test plan. Notice I do not provide a template – the goal here is to get away from filling out a template, and returning to the essence of planning. Once you’ve fleshed out the pieces of your plan, you can go about putting into your organization’s template.

Don’t feel that each of these items should exist in all Test Plans – the amount of planning should mirror the complexity and size of the project, and the details should only match the situations you actually expect to encounter.

“We should think of planning as a learning process—as mental preparation which improves our understanding of a situation.”

· Direct and coordinate actions – define the normal process of testing for your project
o Define the tools that will be used (and how)
o Define the defect process
o Define the test design methodology (scripts, charters, exploratory, a mix)
o Define how test cycles and phases will be scheduled, and how duration will be determined

· Develop a shared situational awareness – This point also covers collaborating with your team (if possible) to draw up the plan
o Define expected metrics
o Define what should be contained within a defect, to support expected and unexpected metrics
o Outline the triage process, with a detailed guide to determining priority/severity
o Outline team responsibilities

· Generate expectations about how actions will evolve and how they will affect the desired outcome
o Define escalation processes
o Define processes to handle environment outages
o Define how builds will be evaluated (smoke tests, procedures if the build is considered a failure)

· Support the exercise of initiative
o Lay out guidelines to determine how much regression will be run, and when
o Establish guidelines when a tester may stop the normal testing process and immediately begin troubleshooting (broken build, application defect causes major failure, etc)
o Establish points in the process where testers can force attention of stakeholders for input (rather than the usual process where testers wait on everyone)
o Establish points in the process where stakeholders are given the opportunity to review and give feedback on test deliverables

· Shape the thinking of planners
o Detail out the likely problems that your testing process will face
o Detail out plans for addressing/mitigating those problems

Our outline just got huge – what a lot of work! Again, not all of these ideas apply to all projects, and many projects will have more facets. Also, plans can be largely reused for the same application when the base assumptions do not change. That said, planning is not just about producing a plan – it’s about investing time and thought into what you will be doing.

If you invest time and effort into planning (preferably with collaboration from your team that will execute the plan), you will find that your team goes in the desired direction, even when things go awry – because the planning process has actually prepared you and them for what they will face.