A/B testing : Delivery time vs Ratings
When it comes to designing a food delivery app, designers are looking for the best solution to ensure they meet the goal, which is, to convince the user to order asap on the app!
With all the extensive research (in the early design phase) in place to understand what converts and what doesn’t, designers often list out a really long list of trends and observations from the design world. Things like changing a button to a particular shade of a color, using a big food image, or employing a certain layout. However, it has been proved over and over again that there is no one size fits all rule to conversion! One needs to keep testing option A vs option B.
About the Zomato app
Zomato, is food delivery app helping people discover great eating places around them. It has a variety of features that makes amazing dining experience for a user. This includes online ordering, table reservations and browsing dining options nearby. Having stated all of that, a major proposition of the app still remains to order food online.
Trigger the action
Here’s the theory of trigger-motivation-ability
The theory states that for a behavior to occur, these three parameters must converge at the same moment. Designers often refer to this model to understand what stops people from performing a particular behavior.
When dealing with food delivery apps, the single point of brainstorming is — What is the one thing that triggers a user to order online? After lots and lots of testing, the food domain has boiled down to the two most important parameters —Rating and delivery time. One often finds food delivery apps playing between the two triggers.
What works better for Zomato?
A/B testing is the best way to understand what is working and what’s not! Just to back up what I just said, Google tested not only the 2 shades of blue, but the 39 shades of blue in between!!
The aim of this A/B test is to understand what triggers a user on a food delivery app, Zomato — Rating vs Delivery time.
Prior to conducting usability tests, I developed a user persona to better understand the target users of Zomato’s android app. This process helped me get into the mindset of the users, thinking in terms of their contexts, needs, and goals
Platform used for experiment
I have used the CanvasFlip online tool for creating the prototypes and for UX insights such as session replay (the user videos). conversion funnel and heat maps.
Number of users in the test
I prepared a list of 12 users. Picked up 6 randomly for the prototype with delivery time highlighted and the other 6 for existing app prototype.
Task given to users
“It’s a food delivery app prototype. Order your favorite dish from the app!
Before getting started with the analysis, try out the prototypes on which the tests were performed
Prototype : Restaurant rating highlighted (Existing)
To conduct a basic A/B test, you first need an existing app design against which you test your variation. So this is the prototype of the existing app design. (Try this prototype in a new tab)
Prototype : Delivery time highlighted (Variant tried)
One of the key aspects of A/B testing is that you change only one variable at a time to test. Here I have highlighted the parameter “Delivery time” against the ratings. (Try this prototype in a new tab)
Usability test results :
I have used the CanvasFlip tool for this A/B testing. I compared the UX insights on the prototypes. To be in a more confident position to state what worked vs the other, conversion funnels and interaction heat maps are quite helpful.
Conversion Funnel of the two prototypes
We do not see a huge difference in the conversion rate of the prototypes. Which is kind of very obvious! A user will not drop-off because he did not catch the attention of the restaurant rating or delivery time. These parameters are highlighted or semi-masked just to make sure that user’s appetite and desire to order is triggered faster.
Still, we do find a higher conversion in the favor of “delivery time” prototype. A/B testing is all about making these incremental improvements. Any improvement in conversion is a great start and puts you on the right path.
Time taken on the app
When we are talking so specifically in terms of the trigger for placing the food order, the time spent on the app matters the most. Unlike an app where engagement and more time spent are brownie points, Zomato is an app where the user wants to move out as quickly as possible.
We see a huge difference in the time taken taken on the two prototypes.
It’s important to understand what matters to a user when he is on your app. As we understand from the usability test data, in this era of always-busy-users, delivery time plays a larger role than the restaurant ratings. Delivering a solid and differentiated user experience is a critical part of survival for the food delivery apps.
Over to you
What do you think about the parameter that should be highlighted — restaurant rating or delivery time? We’d love to know your thoughts. Use the comments section below.
P.S. — If you liked the user experience data used in this experiment, give CanvasFlip a shot and try the same on your app!