Since the public rollout of HealthCare.gov, there has been an unprecedented discussion of the software development process and software quality assurance. Headlines with such mundane terms as “end-to-end testing” have actually been used! One frequent topic that has appeared in these articles is that HealthCare.gov was developed using the Waterfall method. (The Waterfall method is a development process that follows a fixed linear sequence.) So after watching the Senate hearings on the news, will decision-making managers now conclude that: Waterfall = Bad???
Despite the bad press, the Waterfall Method may never go away completely. In some industries, regulatory requirements make it impossible to use anything but the Waterfall method for software. In other cases, organizations may not call what they do a “Waterfall” but simply “the way we do business.”
Whatever development method is used, there are situations where the code must be released by a certain date. The only circumstance that will change the deadline is an “Act of Congress” (pun intended). In the case of HealthCare.gov, there was a hard deadline compounded with an over-ambitious rollout resulting in a markedly limited amount of testing time. This is an unfortunately common situation. A development cycle takes longer than normal, but the release date is unchanged and squeezes QA time. The schedule shift creates a long period of idle QA time followed by a few weeks of intense craziness where four weeks worth of testing is scheduled into a two-week (or less) slot. This is particularly acute in the Waterfall method where all features are to be complete before they are released to QA.
What are the best ways to leverage your idle time to make your peak time less demanding?
Build the Knowledge Base of the Team
If at all possible, involve members of your QA team in meetings which include other departments within the project. Emphasize the importance of being a good listener and taking detailed notes. For different perspectives, rotate different members into different meetings. Sometimes “intent” can be lost during the translation from idea, to requirement, to written test, to found observation. QA should always test against the requirements, but providing members of the team with insight into the code-building process can transform a narrow one-line requirement into a mandate full of context and decision-making history. The bugs written by informed QA members will include fewer false alarms and will be richer in describing what the code is intended to do.
Resize the Team to Fit the Peak Demand
From the seminal book, The Mythical Man-Month, Brooks’ law states, “adding manpower to a late software project makes it later”. This sometimes gets overly simplified to: “Don’t add anymore people to the project because it isn’t worth it”.
In his book, Brooks gives two explanations for this phenomenon:
- New people added to the project need ramp up time.
- More effort is required to keep a greater amount of people in sync and informed.
The important keyword is “late.” If a software project is late, there can still be plenty of time within the embryonic QA stage. Adding more personnel can very much be worth it if there is enough lead time available. Business Analysts, Technical Writers, and other personnel familiar with the project can be drafted for a short stint to work on testing with short ramp-up time. Additionally, the coding time is an ideal period to bring in outside QA consultants who have enough time to familiarize themselves with the product. If the code being tested is extremely late, a trained supplemental team will be even more valuable. Thankfully, the longer delay will enable you to have more time to create one.
Plan for How You Will Handle the Deluge
Revisiting Brooks’ law one last time, realize that keeping personnel coordinated takes valuable time of key resources. The planning burden can be smoothed out by delegating tasks to everyone ahead of time. Features being deployed should be ranked in importance so that the more critical features get the greatest amount of QA scrutiny. Benchmarking scripts should be set up according to spec, and cross-training should occur to eliminate single point-of-failures. Testing documents should be up-to-date. In other words, if you have time to plan – plan.
As we’ve seen with HealthCare.gov, if the software being shipped is indispensable, there will be a follow-on effort. The input provided by the QA team on what is broken will play a direct role in determining what should be fixed first. The QA team will be on center-stage as their input is heavily weighed. During a reclamation/salvage scenario, the bugs submitted and documentation trail created will be used as an authoritative guide amongst the chaos. If the team has effectively used their time, the planning, forethought, and quality of work will shine brightly and get noticed. Spending time in the beginning will pay off in the end.
What quality issues has your team run into due to a lack of testing time? In your opinion, what is the ideal amount of time to test software? If you have any additional thoughts on how to manage a QA time crunch, please share below.
Chris Hasbrouck is a Consultant II at SDLC Partners, a leading provider of business and technology solutions. Please feel free to contact Chris at firstname.lastname@example.org with any questions on this blog post or to further discuss software testing.