5 CRO pitfalls and tips to professionalise

Tip 1: See CRO as a means, not an end 

Employing conversion rate optimisation (CRO) is regularly used as an objective as such. In many organisations, the results are an afterthought. For example, we see that CRO is only applied once and that insignificant A/B tests are still carried out. Changes are then implemented because too much value is already attached to the examined feature, product or design.

During the CRO process, you apply different types of research to examine a particular question or issue. By formulating the correct problem statement, accompanied by a research objective or hypothesis, you can prove whether the idea was really a good idea. 

Does a study not have the expected outcome and is the hypothesis rejected? Then you can learn from this. A negative outcome also offers information to get to know your visitor/(potential) customer a bit better. With well-founded research based on pre-established hypotheses, you avoid drawing wrong conclusions or simply adjusting something based on an idea.

A good CRO process is a continuous process that is part of your strategy to influence certain KPIs. It is therefore not intended to be an unstructured, one-off survey. 

Tip 2. Base your CRO activities on facts, not feelings

If you had plenty of time and an infinite budget, you could validate everything on your website, but this is time-consuming and expensive. In practice, we see that organisations often formulate hypotheses without a problem or probability definition and prioritise them based on feelings rather than facts.

Do you want to get started with CRO? Then we recommend that you first conduct behavioural research and identify the problem areas:

• Where do visitors get stuck and leave your website?
• Which important journeys are there and how do visitors experience them?
• Can certain groups be distinguished?

Next, look at the 'why' question:

• Why are these visitors exiting the website?
• Why do they even behave like this?

Data-driven problems and/or opportunities automatically follow from this. It is important that you then test it for its potential and the organisation:

• What is the scope of the opportunity or the problem?
• By whom and where is this experienced?
• Is there ownership and commitment to the problem or opportunity?
• Does this align with our objectives and KPIs?

In our work, we experience that teams like to come up with solutions as quickly as possible without being very clear about which problem they want to solve and which metric(s) contribute to this. Problems often have sub-problems for which you can come up with stand-alone hypotheses.

For example, do you want to simplify the online experience of your visitors? This can be done in many ways and in many versions. Therefore, you can split the formulated problems and/or opportunities into several hypotheses, in which you can examine several variants.

You prioritise these according to reach, impact, cost and effort. This is how you solve problems or capitalise on opportunities to influence certain behaviour. This way, you take targeted steps to increase your conversion rate and avoid haphazard actions.

Tip 3. When it comes to CRO, look beyond just A/B testing

It is a misconception that CRO only consists of performing A/B testing.

CRO is trying to influence the behaviour of visitors on digital touchpoints to achieve more desired actions and thus create more business value. 

A/B testing is just one of the tools you can use to validate a hypothesis and generate learnings to achieve an objective. This objective can thus be anything, as long as the business objectives and vision remain the focus. For example, think of:

• More sales from a webshop
• More registrations for an event
• More online changes in a service environment

CRO is therefore not only focused on sales. It can be applied anywhere within the organisation. For example, you can improve your service or retention by means of a qualitative usability study. Or by collecting call centre data to discover what the customer is calling about and what your potential customers are up against so that you can provide the right content on your website based on that and reduce the number of calls.

Is an A/B test suitable for your purpose? Then it is important not only to examine behavioural data but also to answer the 'why' question. For example, you then hold an interview prior to the A/B test that allows you to gather insight into visitors' motivations. This way, you ensure that you do not rely on the assumptions you make after only seeing the behavioural data. This will also ensure that your A/B test aligns better the problem or opportunity.

Tip 4. Make sure your data quality is in order

You formed a hypothesis and conducted research. Time for the analysis. Right?

What we often see in organisations is that the data collected is not verified. People blindly rely on what a calculator or self-built spreadsheet produces. A good data check is an important part of any research before you proceed to the analysis.

Try to answer at least these questions for each data quality check:

• Is there a balance between the different types of data?
• Are there huge outliers?
• Do you see empty fields or incomplete data?
• Was the targeting set correctly?
• Is the data actually being collected correctly?

It could be that you are struggling with sampled website data, that your A/B testing programme is not set up properly or that the distribution of an A/B test was not done 50/50, and thus you are experiencing Sample Ratio Mismatch (SRM). So always check your data and have you recently started using a new A/B testing tool? Then do an A/A test to test the tool.

Tip 5. Select the correct statistical test

Experience shows that there is little statistical knowledge available within many organisations. This is one of the most important skills you need to conduct an investigation into accuracy.

For example, A/B/N tests are often viewed with the same statistical test as an A/B test. We also see that a reliability interval is adjusted after the end of the test or the test is discontinued earlier than the duration of the test because a significant result has already been obtained.

Whatever statistical test you apply; in most cases, there will be an answer. To draw the correct conclusions, you also need to have the proper statistical knowledge and research skills.

This starts with a correctly formulated and defined hypothesis. You do not test a complete website redesign compared to the old design, but you test minor differences based on your assumptions. When you put a lot of differences next to each other, you cannot indicate which difference has really made an impact. Also you cannot see what has had no or even a negative impact at all. As a result, you learn nothing from the research itself, which is actually very valuable.

Conclusion 

To do conversion rate optimisation well, it is important to be aware of the pitfalls. Keep the tips from this article in mind and optimise not only your website every day but also your CRO process. This way, you will increasingly achieve more value from CRO, and your activities will genuinely contribute to your organisational objectives.

Receive data insights, use cases and behind-the-scenes peeks once a month?

Sign up for our email list and stay 'up to data':