The quality of web analytics implementations

In practice, we often encounter high confidence in web analytics data, while implementation has serious problems. This often happens when a company thinks that implementing a web analytics tool is very easy. But as Microsoft's Ronny Kohavi once said, 'Getting numbers is easy, getting numbers you can trust is hard.'

'Getting numbers is easy, getting numbers you can trust is hard' - Ronny Kohavi, Technical Fellow and Corporate Vice President Analysis and Experimentation, Microsoft

The reverse situation also occurs. There is relatively little trust in the web analytics data, which is unjustified in that specific company. This situation usually arises when there is a small deviation in the measured goals for the site compared to a backend system. While within the company there is an expectation that web analytics data must be 100% accurate.

Accurate and precise 

Web analytics is not 100% accurate, but it is precise enough to extract important insights and actions for the business.

By 'not 100% accurate' is meant that there are always reasons why this data is not perfect. You won't be using e-commerce data from your web analytics for a tax return anytime soon.

Two examples why this is so:

• People visiting your website with different devices who do not log in will not be recognised as the same user.
• Ad-blockers sometimes block web analytics tools, leaving you with no data at all from some users.

But is a that bad thing? Is the purpose of web analytics to perfectly measure every interaction with your website?

No. The purpose of web analytics is to analyse visitor behavior on your website and thereby improve your website. And for that, your data does not have to be 100% accurate.

Precision is more important. Is there a certain consistency in the data collection, so that we can detect trends? And so does it stand out when certain figures are markedly different from what we would expect from the past? Furthermore, can we extract actions that will help us improve the score of the most important metrics (Key Performance Indicators - KPIs)? That's what it's all about in the end.

Of course, this does not mean that accuracy (the degree to which your web data is accurate) is not important at all. Just how much confidence can you have that your data accurately reflects what happens on your website if only 70% of actual transactions are measured? Or even 125%?

So we should aim for a good score when it comes to accuracy, but don't worry if it's not quite perfect.

You don't get reliable web data by itself 

Unfortunately, it is not particularly evident that a web analytics implementation will deliver accuracy and precision. Because of how easy it is to put basic tracking code on a website, placing such an implementation may seem like a very simple task.

It's not that simple to create something that gives you really useful insights. That is why things regularly go wrong on a certain level. Some examples:

• All websites have goals. Such as selling products, generating quote requests, or posting content that is read. Those goals are usually not expressed in words like 'the page was loaded by the user'. But that is what a standard page view means. So an implementation with just a standard pageview automatically means that you will not get the value out of your analytics tool to the extent that you could. After all, this does not yet measure whether an opened article has actually been read, or how many products you have sold.
  
  
• There is Personally Identifiable Information (PII) in some URLs and it is not filtered out, or only in the Google Analytics Admin interface. The former in particular is ethically not OK. In addition, it is problematic with regard to the General Data Protection Regulation (GDPR) and also violates Google Analytics' terms and conditions. If you save PII, Google reserves the right to simply delete your account. The latter even applies to filtering personal data too late (with Google Analytics filters), because the PII then still ends up on Google's servers.
  
  
• If you're evaluating marketing campaigns, you don't just want to know how your goals are being met, but also what campaign brought someone to your site.To do that, you also need to know what you are doing. When you don't know this precisely, you can reach a situation where you would be better off throwing away all your data and starting over. 
  
  
• Many sites today do at least have components that fall into the Single Page Application (SPA) category. Dealing with SPAs in the context of web analytics is difficult. Chances are things will go wrong here.
  
  
• The web analytics implementation was once well done, but due to changes in the website's code, things broke down a while ago, and that has not yet been noticed. 
  
  
• Most analytics tools require e-commerce tracking to have a very specific structure when sending data. This is where mistakes are very often made. For example, a single transaction that enters analytics several times (sometimes even more than 10x). Or extra product dimensions that have been set up, but never arrive in reports because they are sent incorrectly.

Research into the quality of Google Analytics implementations 

Brian Clifton @ Superweek Analytics Summit 2019 

Brian Clifton spoke during Superweek about his research into the quality of Google Analytics implementations. His focus was not only on GA because he was once the first Head of Web Analytics EMEA at Google, but also because Google is the most widely used in the market. In the Digital Analytics industry, we often encounter implementations with problems or even downright bad Google Analytics implementations (both with the free version and the paid version of GA).

The survey involved 75 commercial websites, all brands that are leaders in their category. 13 of them operate worldwide. Each and every one of them are companies with hundreds of thousands or even millions of visitors to their website every month. Three of them even have more than 100 million visitors per month.

Data Quality Index 

Brian scored them on several points in the categories:

• data policy & legislation
• marketing & acquisition
• custom measurements
• conversion measurements

He arrived at a data quality index per company, with 100 being the maximum score.

Of course, you hope that large companies that invest a lot in web analytics will have their affairs in order, but unfortunately, no they don't. The average was 35.7%, a huge deficiency. But 9 companies scored above 50%. The lowest score 4.5% and the highest 73.1%. Even these market leaders analyse the behavior on their website with a (very) mediocre implementation.

1 in 5 companies store personal information in Google Analytics 

Brian showed an example of a report of URLs with email addresses and passwords of logged-in users in them. And yes, we have experienced this in practice at several companies (and also a combination of name, address, and credit card number).

Some companies send all the values people fill in on a form to GA. In most cases, submitting PII to an analytics tool is something that happens subconsciously. A clear sign that the people working within them do not have sufficient knowledge and understanding of what they are doing.

A few more interesting results of the study:

• Only 19 companies had the transaction measurement well in place. While 31 are e-commerce operations.
• The topic that scored the worst was visitor segmentation (85% scored 'wrong or missing'), while the use of segments is the most important thing a web analyst has to do to extract information from his data. 
• Campaign tracking is also an important part of web analytics, which we also wrote about before because it is often done wrong on some level. That turned out to be the case again. However, 18% of the companies in this survey had done this well.

Take a critical look at your web analytics implementation 

There is clearly still too little attention paid to the quality of web analytics implementation at many companies. Pretty crazy when you consider that:

• for more and more companies the website is the most important point of contact with their customers;
• many companies run experiments on their website and look at the web analytics data to evaluate those experiments;
• more and more companies (among others) want to use the same online data to personalise the customer journey.

Do you have a good idea of the reliability of your web data? Do you know where improvement is needed? Brian Clifton has automated his check and developed Verified Data . We have tested this tool and decided not to use it in our Google Analytics audits. 

A good analytics implementation is tailor-made for a website. An automated check only gives a rough indication of how things are going.

We like to give practical advice where we don't just look at general improvement points. We provide specific explanations about the issues that we find. That way you know what to tackle. And that requires a manual audit.

Do you want your web analytics implementation to be thoroughly vetted?Contact us. 

Receive data insights, use cases and behind-the-scenes peeks once a month?

Sign up for our email list and stay 'up to data':