Quick A / B Testing report (UX)
Deliverables:
- Usability testing
- Come up with strong feasible variations of hypotheses and design them
- Run an a/b testing
- Data process and analyze
- A full report on the test
Goal: To improve conversion rates from trail visitors into customers.
Results:
MSA was able to rapidly increase conversion rates from visitors into customers by 20 %
Background:
After users signed up for the trial version most of the users fail to renew and pay their premium subscription because of the lack of features the app has that actually retain the users
As well as the usability problems, the app’s visual design didn’t meet the standards that their clients expected.
MSA CEO asked me if I could help out with a redesign. We decided to spend 6 months working together to improve the platform’s design.
Fast usability testing:
Part of my goal was to identify and solve critical usability issues. I sat down with Ben to identify the most important areas of the product to focus on, using the features that current clients were using most as a guide.
I wrote usability test scripts around important parts of the product and tested them with 5 users. This would be a cheap and effective way of identifying the most salient issues.
The results were worse than I expected — most users completely failed each task, and those that were successful typically took a long time and had to use more clicks than necessary.
Usability evaluation:
I decided to combine a usability evaluation with usability testing, to get a better picture of where users would be failing tasks.
Typically usability evaluations benefit from more than 1 evaluator, but I felt that we would catch the most serious problems with the usability testing, and there were enough minor problems I could identify to give us plenty to do.
Additionally, I gave each participant a survey after they completed the test, to get a sense of how the app compared with other streaming apps. The score was 74 — an average score, but with room for improvement.
In general, I noticed that users thought the app wasn’t that difficult to use, but also the lack of features that simplify benefiting form app wasn’t there
so our first action was to research and find out additional features that make the app easier to use and also has its unique value in the market
A/B Testing:
After creating some new home page alternatives we had doubts what’s the best version to stick on . all the alternatives had the same features with a different style and layout.
So first, We had to answer these questions:
Which copy increased trial sign-ups?
Why do we need the a/b testing?
Which Landing Page got more lead generation from submissions
Does matching headline & body format on the pages matter?
Does adding new features to your home page increase sales?
Does clean minimal design increase conversions?
Which page got more leads?
Did the page get a lift due to a free account sign-up?
Which page drove more clicks to a lead generation form?
KPIs
Those are the KPIs we set up in order to decide which copy perform the best
- page view
- median initial loading time
- sessions
- conversion rate
After running the a /b test for 90 days and analyzing the data and calculating the results we had a winner
This is the desired home page:
