New platform improvements through A/B tests
2019 - UX / UI / AB test for conversion rate increase
Background
New casual site is our new mobile games platform, with the existing 3 values of our branding - “Trust, Value, Speed” - as the cornerstones, providing better experiences to cater to our target audience while making the product identifiable.
Current problem is that the conversion rate from login to purchase on new platform is less a lot than on current platform.
Goal
Increase conversion rate(login-to-purchase) from 22.5% to 50% through A/B tests
Our team’s goal is to reach 50% of the conversion rate of new casual platform within one month, to help users find the items that they want and purchase more on our new platform.
Use data-driven approach to A/B testing to make decisions and give the trustworthy conversations with stakeholders.
My role
- Initiated A/B testing to validate assumptions and continually iterate.
- Collaborated with our design team to generate solutions and design
- Worked with PO and engineers to deliver designs and run A/B testing
- Collect the result from our database and Google analytics, and reported to PO and stakeholders to make final decisions
Break down user journey
We listed down the user journey to figure out the potential problems from login to purchase and understand the click rate for each actions for us to analyse what are the differences between new and old platforms. These were definitely the key points we could improve first.
Iterate
After sharing the key points and brainstorming with PO and teams, we started our A/B test iteration.
Averagely, we worked with engineers and delivered one idea for A/B test within 1-2 days.
Below are some of our improvements from design to validation:
Outcomes
After launching these improvements one month, the result was in an overall 70% increase in the conversion rate of new platform.
Although we didn’t reach our goal, the value and effort of our A/B testing were recognized from managers. In addition, we established a A/B test process of quick validation and iteration.
Lesson learnt:
1. Prepare measurement/tracking plans to discuss with PD’s/PO before creating new project
2. Use global variable to distinguish between pages to check data from different groups
