Chances are your test data is costing you more time and money than you know.
Like most organizations focused on quality, you probably spend considerable time, effort, and most importantly budget to ensure that your software has the highest level of quality. Despite your best efforts and investment, if you are still seeing defects in production, project delays, and testing taking longer than planned, test data may be a primary culprit. Industry research suggests the time spent on test data related defects amounts to 30% or greater of total testing time. An SDLC Partners survey of our QA professionals across numerous customers in various industries showed that nearly 55% spend between 5 to 15 hours per week in setting up or addressing issues with test data. That equates to at least one day lost per week due to test data challenges!
From knowing exactly what data to test, identifying the correct volume and variety of test data, determining the optimal process of sourcing test data and sharing it across testers, managing test data is critical to the success of your testing process and ultimately the quality of your application. Sadly, we have seen less than 25% of the organizations we worked with have a test data strategy in place, and even less are able to execute on it. What we see most often, is organizations making a production copy of data for testing, and then the testers hunt and peck for the needed data to support test execution.
This process is far from optimal and can lead to dangerous consequences.
Imagine that your customer receives an email from a test system because a test scenario called for email testing and the test data used production email addresses. Now imagine if that test email contained sensitive data such as the customer’s account number or social security number, or even protected health information.
Now, envision that your new functionality requires very specific data to validate it, but there is nothing currently in production with that capability. After much work, most testers will “tweak” production data until they feel it covers the scenario. You might be able to get away with this the first time, but ensuring proper test data for ongoing consistency is critical. While these situations are not optimal, one of the top issues we see is the dependency and lack of a test data plan from integrated or supporting systems. These days, most systems are heavily linked and connected with internal and external 3rd party systems. Proactive planning to ensure that they are ready to send you the right test data, when needed, is critical. These are just a handful of the challenges that a Test Data Management (TDM) program solves. TDM is defined as the process for identifying, provisioning, sharing, staging, storing, and governing test data. TDM can be implemented with varying approaches to best meet the organization’s needs. For some it is purchasing and implementing a commercial tool – usually including a hefty price tag. For others, it is implementing a new testing organization staffed with test data tsars who approve test data requests and coordinate the provisioning of test data across all organizations – often leading to bureaucracy and frustration.
Each of these has its own pros and cons, and developing the right program depends on each organization’s unique set of challenges as well as their goals, existing technology use, and capabilities. Smaller and leaner organizations may not need a fully-staffed centralized TDM team, but might benefit from better automation of their manual processes for test data generation. For larger organizations, it is critical to have a test data strategy in place and a staff that can facilitate the execution and ongoing management of that strategy. For larger organizations with numerous systems and testing cycles, the proper management and governance of test data and test environments is a huge factor in the success of the project.
Finally, for organizations of any size, being able to fulfill test data requirements in an agile environment ensures writing requirements faster, developing faster, and testing faster with automation, without sacrificing quality.
As we have highlighted, good TDM is not just a buzzword, a methodology, or a tool you can simply buy. TDM requires you to have a planned strategy and program, and knowledgeable employees that understand the domain well enough to know what test data is needed. A strong TDM program also requires that you have a process you can follow for identifying, provisioning, staging/storing, and executing with test data in a repeatable and managed/governed fashion. Finally, a good TDM program also utilizes technology to mechanize processes, automate monotonous, repetitive steps, and build synthetic test systems to remove dependencies, so that your testers can focus on the critical functional testing needed.
Software quality is a huge competitive advantage in today’s market with ever-more demanding customers. Good test data management can be a key factor in increasing your speed to market while delivering high quality, thus directly affecting your organization’s success. By leveraging a combination of a solid strategy, people who can quickly obtain domain knowledge, automated tools that reduce manual effort and improve data quality, and best practices, results will be achieved with both time and cost savings. For more information on the value you could receive from better test data management, contact SDLC Partner’s Software Testing and Quality Assurance experts to design a TDM program that is just right for your organization.