Extending a framework of automation testing into an existing regression effort can easily save many dollars and man-hours that could be expended on more mission-critical priorities. During my experience developing Mobile Automation Testing Architectures, I have aggregated and generalized some important principles of automation from the mobile device context. In this two part series, I have compiled 7 tips and best practices that I have learned along the way. Without further delay, I present the first 4 tips…
1. Premeditate Your Architecture
Careful consideration should be given to the design of your testing architecture. As your automation scripts begin to cover an ever increasing range of functional areas within your website or application, the increasing complexity will be costly and time consuming to manage if structure, coding standards, and organizational conventions aren’t designed and followed from the start. Creating such a system is easy to do, learn and understand, and will increase the efficiency of long term growth and maintenance. This will keep your automation suite robust and portable – which is increasingly more important.
It is easy to devote too much time to this phase, so keep this in mind if you are adhering to a timeline. Determining the level of design should be discussed and agreed to up front. There will always be issues that only reveal themselves in practice, so it’s important to balance the depth of design in this phase with getting something ‘out there’.
2. Get Something ‘Out There’
It is doubtful in any automation effort that an individual, or even a group of individuals, will be able to preconceive every possible design element the architecture will require. Writing a few skeleton scripts and beginning to automate simpler tasks will prove an effective means of testing the initial design. During subsequent phases, necessity really is the mother of innovation. Problems will present themselves which might scrap your original design, or change it for the better, so be accepting and flexible.
Use the skeleton scripts you’ve built to prototype for expansion of your automation coverage. You might find that following your initial design would lead to an unmanageable mess. Going back and forth in initial phases will achieve two important goals; your architecture will be clean, manageable, and ready to grow, and you will have a solid framework for the rest of your automation coverage.
3. Separate the Impractical
After the architecture is designed, decide which areas should fall under automation testing and which areas are better left to manual testing. The best candidates for automation testing include: highly repetitive monotonous tasks which are prone to human error (such as clicking a link and verifying content), data entry tasks (such as form submission), as well as most functionalities tested during regression. Typically, tasks which can destroy data or require a keen eye for detail should be left to manual testing.
4. Maximize Coverage, Minimize Devices
As of November 2012, the iPhone operating system iOS made up 48.1% of operating systems on all mobile devices in the United States according to Kantar Worldpanel, a leading business & consumer intelligence company. The Android operating system similarly held a 46.7% share of the U.S. market. By strategically dividing a testing effort among a combination of iOS and Android devices, one can test up to 94.8% of the population with these two operating systems alone!
This makes two things clear: a majority of testing should be performed on iPhone and Android devices, and testing on any other OS could potentially expend many resources without addressing a large enough user-base to justify those increased costs.
Deviation from the general distribution is normal in the localized population of users who will actually use your mobile website or application. For example, the Windows Phone operating system accounts for less than 5% of the mobile OS market share, however, if the largest segment of the population who uses your particular app or mobile site is running Windows Phone 8, then the testing effort should certainly include this operating system regardless of its relatively low market share in the broader population. Evaluation of how users are accessing your mobile application should heavily influence the distribution of testing on a case-by-case basis.
Now that’s a lot of information! Take some time to think about these four tips and then come back next week for the final 3 best practices in SQA mobile automation.
Joshua Sawyers is a Consultant II at SDLC Partners, a leading provider of business and technology solutions. Please feel free to contact Joshua at firstname.lastname@example.org with any questions on this blog post or to further discuss mobile automation.