Comparing apples to oranges…same juice, different taste

As the web keeps turning from being an extension of a business to the driving force behind it, more and more tools are made available to track visitor behavior.  With this comes the “apples to oranges syndrome,” which occurs when you start comparing data from two different systems and expect them to match.  The one occurrence that I keep seeing is the comparison between an analytics tool to the client’s ecommerce system.  Before I continue, I’ll just give you the punch line:  THE DATA WON’T MATCH!!!!!!      

Now that we got that out of the way, I’ll start explaining why the data won’t match.  When looking at a tracking tool, whether is an analytics tool, or a campaign tool, or your company’s backend system, each have its own set of moving parts, with some parts set in such a way that even though they might look to be tracking the same data when comparing to another tool, in reality they’re not.  Here are a couple of reasons that explain this phenomenon:

·         Unique architecture – as tracking tools go, each is created with its own set of algorithms and definitions, thus rendering them, as well as the data that is presented unique.  This can be seen when comparing data from multiple analytics tools.  Since each tool collects, process and presents data in its own way, what one tool considers quantifiable, another might not. 

·         Metric definition – as well as architecture, each tool defines certain behaviors in its own fashion.  What one tool calls a page view, another might call it a hit, and another might call it impression.  Each metric definition is placed because it fits best with the tool’s objectives and is more relevant to what it reports. 

·         Cookies – cookies play a big part in tracking visitor behavior because not only it track specifics about where the visitor was and where it’s going but it recognized a visitor if it leaves and decides to return (provided that the visitor accepts the cookie and doesn’t delete it after leaving).  Although the trend is to use 1^st party cookies because the data that it collects is more accurate and is less likely to be blocked by visitors, there are tools that by default uses 3^rd party cookies.  The problem here is that if you use a tool that uses 3^rd party cookies and another that uses 1^st party cookies and a visitor is blocking 3^rd party cookies (which is the default security setting in most newer browsers), the tool that uses 3^rd party cookies won’t track that visitor. 

·         JavaScript – most tools require you to place some JavaScript code on the site in order to collect traffic data.  Depending on what tool you’re using, it might provide you with a noscript tag that is placed on the page as well.  This ensures that if a visitor is blocking JavaScript, that its behavior is tracked.  If you have multiple systems but only one uses noscript tags, then that tool will be the only one that collects data for visitors who block JavaScript.

·         Security settings – security settings affect how different tools accept incoming/outgoing traffic and collects data.  One example of this is IP blocking.  If your company doesn’t want to track traffic that comes from certain IP addresses (like internal traffic, for example), some tools allow you to block them.  Another item to consider is that some firewalls are set to block traffic to certain domains, which affects how a tool tracks traffic.  For example, say that an employee in your company decides to buy one of your widgets from his work computer (yes, we all are guilty of that) and your analytics tool blocks all internal IP addresses.  When you look at your sales report and compare it to the analytics report, that visit and everything that pertains to it was not recorded. 

Now that you’ve read this, you might be thinking that they’re all useless.  “Not really”, I must say.  Understand that each tool provides a unique representation of your site’s health and how visitors react to it.  Rather than trying to figure out why the numbers don’t match and dismissing them altogether, take a step back and look at the bigger picture, look at the trends in them.  If you see that the trending behaves in similar fashion, you know that each tool is working as it should.  Remember that it’s very difficult to capture every single page view, and multiple tools won’t give you an exact representation, but they’re pretty close to the truth.