By combining quantitative and qualitative datasets, you can improve customer experience through effective experimentation.
Conversion rate optimisation (CRO) is the process of increasing the percentage of users on a website or mobile app to fulfil a particular business goal. Many CRO techniques and practices are utilised in this field to optimise the user experience and can be categorised into either research or experimentation.
Typically, the latter plays a heavier role, but research is key to testing and optimising for a holistic data-driven approach.
It’s an iterative process
What does the research and experimentation breakdown look like? For a fluid approach to CRO, an iterative process is required; one that consists of steps to be actioned to validate a hypothesis.
This can be broken down into the following phases:
1. Observation: What is the problem you are trying to solve? What business KPIs are you addressing and across what parts of your website or app?
2. Research: Quantitative and qualitative analysis to understand the problem and identify opportunities for implementation and experimentation
3. Hypothesis: Based on research, create actionable hypotheses to predict the outcome of the experiment
4. Experiment: Test the hypothesis in the live environment (A/B testing is the process of comparing two or more versions of a webpage or app against each other to determine which one performs better)
5. Analysis: Review experiment data to understand the impact on key metrics
6. Conclusion: Review next step opportunities; implementation and/or iteration
What are the benefits?
The primary benefit of CRO is the quantifiable return on investment (ROI).
Running A/B/n tests will give you conclusive statistics on uplifts against a control group. This concrete analysis differs from other business units that don’t allow for an exact measure of success.
A CRO program offers additional benefits to quantifiable ROI, such as:
- Deeper customer insights: expand and cement your understanding of how your audience behaves through top-to-bottom auditing and experimentation
Continuous optimisation: through data-led insights based on research and experimentation, your website and contents can be constantly improved and refined by a structured, iterative process
Real-time results: optimal performance of product launches and marketing campaigns arise from real-time data-led decisions as opposed to pre and post-campaign analysis, removing the element of doubt from your business decisions
Always do your research
To run an effective experimentation program, you need to understand your audience. i.e., what are the customer’s intentions and how are they flowing through your website? Therefore, a combination of quantitative (what is currently happening) and qualitative (why it’s happening) research is required.
Quantitative research: what is happening?
Typically, your quantitative research will come through your web analytics platform. Your aim should be to understand lay-of-the-land performance including conversion rates and engagement metrics, overlaid with a variety of dimensions such as channel, device and page performance. From there, you can begin to focus on specific pages and explore funnel performance to identify optimum and underperforming journeys.
Qualitative research: why is it happening?
Heatmapping will help to bridge the gap between quantitative and qualitative analysis and offer insights into how customers are interacting with your content and their potential friction points. These insights plus session recordings, polls/surveys, user testing, competitor research and further heuristic analysis, will give you a holistic understanding of audience behaviours
Using your research to inform experimentation
Once you’ve collated your research, you can start forming hypotheses for A/B/n testing. Depending on your hypothesis, you may want to test changes such as messaging, imagery, content order or complete page redesigns.
For testing with multiple variables, the option exists for multivariate testing if the traffic volume is there. This will reduce the need to run iterative experiments but requires significantly higher volumes of traffic to be successful.
All experiments should be deployed and monitored through a suitable A/B testing platform and left running until the data has achieved statistical significance - the point at which your result is guaranteed to be a success or not.
Back to the research
Research and analysis are not limited to the start of a CRO program. The iterative nature of CRO means ongoing research, particularly when analysing results.
When analysing A/B/n tests, regardless of whether they were successful or not, consider the following criteria:
- Key metrics and conversion rates: detail every key metric relevant to the test, while noting that success to a primary metric might be a detriment to another. Ensure all possibilities are explored, including page flows.
- Additional dimensions: a test may be successful overall, but does this success translate to a specific device? How about for new vs returning users? Demographics? Review all dimensions that could be impacted when segmented out to take advantage of further opportunities for building personalised experiences.
Consider documenting all this information in a report that includes: experiment observation, hypothesis, statistically significant results and next steps.
In summary
Only through the iterative, ongoing process of research and experimentation can genuine CRO be achieved. When implemented successfully, CRO will optimise the user-journey and improve ROI. However, changes made to conversion funnels without data-led research can prove disastrous.
The technical and practical processes involved in performing this research can be daunting if you’ve never done it before, so we’re here to help. Our team of optimisation experts can help you to create a truly effective CRO strategy, so contact us today.
Google Optimize is sunsetting on 30 September
Which alternate provider should you go with? The Conversion Rate Optimisation (CRO) Experimentation Matrix allows you to easily compare vendors across the complex landscape.
