Stepping into the world of Mobile App Testing

Introduction

The mobile industry continues to grow and while apps are becoming extraordinarily adopted, it is still unclear if they deserve a specific testing approach. From combating the mobile testing challenges and to driving testing strategies, there are some interesting stories to tell. In my article, you will learn how to overcome mobile app testing challenges and to ensure the highest quality of your apps.

The world of mobile application is growing rapidly and 87% of the global population are now mobile subscribers.

Introducing new mobile devices in the market introduces tremendous challenges for mobile development companies to provide intuitive and engaging user experience with flawless functionality.

The role of the mobile application testers is very prominent as they ensure that the application works as expected on the various mobile platforms. What follows next is the experience I gained in mobile app testing. We will dive into various areas of mobile app testing from the manual testing point of view, which I found to be very effective when it comes to "raising the bar for quality".

Stepping into the World

“Mobile app testing is about inspecting and adapting to how we test based on empirism”. We are aware of the significance of "Empiricism" in Scrum as Scrum is an Empirical process which means "working in a fact-based, experience-based, and evidence-based manner" which is not alien to Mobile app testing world. In Mobile app testing, we respond to the changes by continuously evolving our test strategies, ways of working and our testing processes.

Platform Specific to Cross-Platform Test Approach:

I began my career journey in mobile app testing as a cross platform mobile app tester coming from the web application background. One of the earliest challenges I faced was: “We had 2 platforms iOS and Android to test”, and the question which lies in front of us was "How should we effectively test both the platforms and still be on time and resource efficient?" When we began, we took Platform Specific Testing Approach which is quite a standard approach followed in the mobile industry, but very soon we found challenges with this approach. In platform specific approaches, the testing teams are prone to falling into working in their own silos, communication and collaboration challenges might appear, misalignment in the testing approaches and strategies is obvious. In order to overcome several challenges of platform specific teams, a cross-platform team approach was adopted, the idea of the cross-platform testing approach is that each tester will test on both the platforms (iOS and Android) in parallel. This approach brings the focus be on “behaviors instead of implementations”.

After adopting Cross-Platform approach, several benefits were experienced:

• Increased collaboration among the teams

• Teams shared frequent best practices

• Standardized approaches across the mobile platforms for writing Test Cases, Ways of Working, Standardized Test Strategies

• New design enhancements/improvements

• Consistent UI could be maintained across the mobile platforms.

Group Testing Sessions:

The next important approach that can help the mobile development team/teams to enhance the quality of the product features are "Group Testing Sessions" that helps to enhance the Mobile User Interface and Usability. Before every feature is released, mobile app testers collaborate with the external project/product teams who are not directly related to the project.

For example, people outside the development team and from external teams are allowed to provide their input and guidance to optimize the product UI/UX /Usability experience. The external teams that take part in the group testing of the mobile applications bring with them an independent approach and user perspective to testing the product, and as a result we could find new bugs, new UX/Feature enhancements, usability issues. The Group Testing session approach works wonders for your mobile product quality and this approach also might trigger a new product discovery.

Exploratory Testing Approach:

We went beyond the group testing approach and added exploratory testing as the part of our mobile testing process which primarily focused on the “risky areas of the app” and was very much “targeted”. The key difference between group and exploratory testing is: In exploratory the members who take part in the testing is familiar with the product whereas, in group testing the external team members who take part in the testing are unfamiliar with the product. Exploratory testing plays a key role in digging out more issues and bugs in the mobile space. In Exploratory testing, we incorporate lessons from past sessions as every such session has something new to teach and we continue to develop our exploratory test approaches.

The exploratory testing process looks like this:

• Initially testers plan for the sessions

• Tracks the testing progress

• Capture the results of our work

• Share the results with the product and dev teams who provide their feedback.

• Based on the feedback received, the new enhancements/bug fixes are brought to the next feature/product iteration.

Mobile App Analytics

One of the biggest known mobile app industry challenges is “Device Fragmentation”

Every year Device Fragmentation Growing 20% Per Year and some 1000s of mobile device are added to the device market pool.

For the mobile application industry, selection of mobile devices, operating systems, and models for testing is the biggest identified challenge. So how do the development teams deal with the device challenge? Some third-party tools can prove very helpful in overcoming these challenges. Mobile App Analytics are the tools to consider which can provide clear stats on the “device, “model, OS” that your customers are largely using based on the demographics and helps drive your device testing strategy. These tools also help with details such as features adoption rate, happy flow that our customers are taking, app usage rate, how your customers are currently using the app and many more and, accordingly device strategy can be identified for the current testing phase and the stats are continuously monitored over time for every product iteration or prior to the release of the app.

“Mobile App Analytics” tools acts as a genie and a key source of information for strategic decision making around app testing. I also recommend testing on the actual devices in 80% of the mobile app testing - teams do use virtual software’s emulators/simulators for their tests, but testers gain more confidence on their tests when the tests are conducted on the physical devices.

Evolving Test Strategies:

In the mobile domain, it is very important to keep evolving the test strategies as we are constantly learning from the previous projects and app release experiences in each project iteration. A lot of information for testers to identify strategy comes from User Interface Research, discussion with the Product teams, Requirement Documents, Story Mapping Workshops, Scrum Ceremonies. The above areas prove very informative when defining a test strategy for a future app release. After every release, it is ideal to conduct a release retrospective as this approach helps in identifying areas which could be improved. QA’s work with the QA leads, Release leads and Development Managers and incorporate new ideas in our strategies going forward. We should always remind ourselves that

“Quality is not just the responsibility of the person wearing the QA hats, we all need to wear it, from the code we write to how we test it.

Accessibility

“So, we are building the apps but, Do we understand our users? And are we really concerned about every individual user of our app? Accessibility is a very important factor in developing the app in the current world.

Our ambition should be to develop the “One Application for All”.

Our aim from the beginning should be to ensure that our customers can support their users with a range of abilities including limitations in vision, mobility, hearing, and cognitive capacity.

We should make sure that our applications are: Perceivable, Operable, Understandable, and Robust for the users of limited abilities. We must build accessibility into our mindsets and we should start to think about how anything we do, and anything we create, can affect accessibility. The most important stage in the life of a development team is when the app is released to the customers.

Shorter Release Cycles

Mobile apps have a release management process which is an enormous domain. We tried moving from a four-week app release cycle to the two-week release cycle. We identified some benefits out of this approach. Shorter release cycles helps to deploy the bug fixes/changes quicker to the customers whereas, longer release cycles or release trains can cause longer delay in feature releases/bug fixes. In order to combat the challenges releasing, we recommend the below procedure:

• Continuously Monitoring the adoption of the released app on analytic tools.

• Monitoring the user responses to the app.

• Monitoring the mobile app store reviews.

Mobile Staged Roll-out's

Monitored staged rollouts of the app is one approach which proves to be very important to mitigate against unforeseen circumstances. The key benefits of the Staged Rollout is that we can understand the customer adoption response and monitor crashes in the app earlier in the release. This approach helps to avoid any bigger impacts to the business.

Learning and Development Hour

The development teams invest an incredible amount of talent building outstanding applications. The technology changes with the speed of light and mainly with Mobile when there are many approaches to development and testing, and how should we ensure we are getting sufficient time for our self-development and growth while fulfilling our career obligations?

In my team we have taken the “innovation days approach" where we allocate a few hours each week to work on our self-development, exploring automation tools, cross-team trainings, self-training courses, independent projects - Innovation of any kind is allowed during these hours. I would highly encourage your teams to try this approach as it brings greater career self-satisfaction and fulfilment along with achieving learning and development goals.