COMPUTERS: Always Analyze an Analytic Analysis
A lawyer presented me with the following situation; two doctors in practice together for 15 years terminated their relationship. Each doctor set up his own office. One of the doctors created a new website with a link from the old website. Their dispute
A lawyer presented me with the following situation; two doctors in practice together for 15 years terminated their relationship. Each doctor set up his own office. One of the doctors created a new website with a link from the old website. Their dispute hinged on a correct interpretation of website analytics.
During the discovery process, the lawyer for the Plaintiff (Mr. Jones) presented a report produced from a popular website analytics program. It was offered as a factual representation of website traffic. The report listed several items, but two were critically important; the total for “visitors” was 12,468 while the total for “unique visitors” was 2,388.
I was engaged by the defendant’s lawyer (Mr. Smith). He asked me, “Can you explain what this report means?” I began by saying, “The report analytics do not mean what the Plaintiff’s lawyer asserted they mean.”
Skeptical, Smith asked me, “Is the software analytical program the Plaintiff used typical for the industry?” I replied, “Yes.”
Does the report state, “12,468 visitors visited the site?”
“Yes.” I replied.
Smith asked, “Then, why do you disagree with the report?”
“Because the report failed to distinguish between human and non-human visitors”, I replied, “And the issue before the court depends on the number of human visitors, not non-human visitors. Non-human visitors do not need medical advice.”
I explained that search engine programs called “robots”, from sites such as Google or Yahoo (and more than 100 other similar programs) regularly visit websites. This process is called indexing. These search engines are non-human visitors. A computer does not distinguish between a human or non-human visitor and counts them both as a visitor.
I then proceeded to show Smith how I could examine the raw server data, identify the non-human visitors and subtract them from the visitor count. The human visitor count was actually less than 1,200; not the 12,468 in the Plaintiff’s complaint. I showed Smith the raw data and highlighted the visitor traffic that came from Google and other search robots. I also showed Smith (by an analysis of the website analytics) how many hours each robot was at the website indexing pages. It was an eye opener.
Smith understood and asked, “Can you explain the 2,388 unique visitors?” Using the same technique I had demonstrated earlier, I subtracted the non-human unique visitors. Then I explained that a unique visitor in analytic reports was a function of the analytical time frame, which is typically 30 days. For example, if I visited the website 5 times within a 30 day period, it would be reported as five visitors; but only as one unique visitor. If I continued that visiting sequence for 12 months, it would be reported as 60 visitors and 12 unique visitors, even though the visitor was only me.
The implications of these data are broad and can be used by website operators to inappropriately inflate the number and quality of their visitors. By inflating the number of visitors, website operators can charge more for advertising or assert they have more prospective customers than they actually have. I have witnessed similar misrepresentations in various disputes and typically it is not immediately clear whether the misrepresentation is deliberate or merely naive. An examination of the website owner’s conduct usually clarifies this issue.
As we continued to examine the Plaintiff’s website report submission, I explained common misunderstandings about hits, page views, click-thru’s, banner ads, affiliate programs, affiliate revenue sharing, server logs, IP addresses, keyword purchases, Google places and CPM advertising.
As I left the attorney’s office, I reminded him to always analyze an analytic analysis and to call me with any questions.