The Executive Dashboard and the Trinity Mindset

JL12345

Web_Analytics_An_Hour_a_Day_----_Chapter_1_Web_AnalyticsPresent_and_Future.pdf

Home >Computer Science homework help >The Executive Dashboard and the Trinity Mindset

■ W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

Web Analytics— Present and Future

On March 20, 2007, a search on Google for

“web analytics” + definition returns 642,000

results in 0.11 seconds. It is a testament to the

complexity and long history of this wonderful

topic (and to how fast Google can return results).1

30652c01.qxd 5/1/07 12:47 PM Page 1

Kaushik, Avinash. Web Analytics : An Hour a Day, John Wiley & Sons, Incorporated, 2007. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/apus/detail.action?docID=302273. Created from apus on 2020-09-07 12:28:37.

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

The Web Analytics Association (http://www.webanalyticsassociation.org) has recently proposed a standard definition for web analytics:

Web analytics is the objective tracking, collection, measurement, report- ing, and analysis of quantitative Internet data to optimize websites and web marketing initiatives.

The dawn of web analytics occurred in the 1990s. However, the preceding definition—the very first standardized definition—was not proposed until 2006, a reflection of how young the field is.

A Brief History of Web Analytics

At the birth of the Internet, things were relatively simple. One would type an address and a Uniform Resource Locator (URL), a file with text and links would be delivered, and that was it. Life was simple.

It was discovered that sometimes errors occurred and the files would not be served or that the links were incorrect, causing a failure. At that point, a clever human discovered server error logs and leveraged them to find information about hits made on the web server (quite simply at that time, a hit equaled a request for a file).

These server logs were capturing not only the fact that someone hit the website, but also some additional information such as the filename, time, referrer (website/page making the request), Internet Protocol (IP) address, browser identifier, operating sys- tem, and so forth. Things started to get cooler because now you knew something about where the hit came from.

As log files started to get larger, and nontechnical folks started to become inter- ested in data, yet another clever human wrote the first script that would automatically parse the log files and spit out basic metrics (Figure 1.1). Web analytics was officially born.

Figure 1.1 A sample report from Analog, version 0.9 beta

30652c01.qxd 5/1/07 12:47 PM Page 2

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ A

B R

IE F

H IS

T O

R Y

O F

W E

B A

N A

L Y

T IC

Analog, written by Dr. Stephen Turner in 1995, was one of the first log file analysis programs that was widely available on the Web. It is still one of the most widely used web analytics applications and it comes installed on websites from most Internet Service Providers (ISPs). Analog, and tools like it, fueled the adoption of web analytics beyond the Information Technology (IT) team. The reports started to get pret- tier, and of course marketing folks could now finally understand what was happening.

Around 1995–96, the general users of the Internet started to get exposed to web statistics because of the proliferation of a delightful thing called a counter. Page counters were perhaps the first example of web viral marketing (credited to a company called Web-Counter). Counters were everywhere you went on the Web; they stood for both being cool and showing how popular you were.

Commercial web analytics started several years later, with WebTrends becoming its new poster child. WebTrends took the standard log file parser and added improve- ments to it, but even more important, added tables and pretty graphs that finally dragged web analytics to the business teams (see Figure 1.2 for sample output).

Figure 1.2 WebTrends sample report

By the year 2000, with the popularity of the Web growing exponentially, web analytics was firmly entrenched as a discipline. Companies such as Accrue, WebTrends, WebSideStory, and Coremetrics were all firmly established as key vendors, providing increasingly complex solutions that reported massive amounts of data.

Around the same time, web analytics vendors and customers were discovering that using web server logs as optimal sources of data presented certain challenges.

Challenges with using the logs included the following:

Page CCaching by ISP The challenge with caching was that after the ISP had a copy of the page, all subsequent pages would be served from the ISP, and the website log files would not have entries for those requested pages.

Search Robots With the increasing popularity of search engines, search bots would fre- quently crawl sites and leave non-web-user entries in web logs. These entries would be counted in the metrics. Although robot hits could be filtered, it is difficult to keep pace with all the new robots (and they get smarter with time).

30652c01.qxd 5/1/07 12:47 PM Page 3

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Unique Visitors With an increasing number of users being assigned dynamic IP addresses and coming via proxy servers, it became difficult to identify unique visitors, well, uniquely. Vendors resorted to using the IP address plus the user agent ID (user operating system and browser), but that was not quite optimal either. If a site set cook- ies, those were used, but not all IT departments readily did that.

For these and a few other reasons, JavaScript tags (a few lines of JavaScript code) emerged as a new standard for collecting data from websites. It is a much simpler method of collecting data: a few lines of JavaScript are added to each page and are fired off when the page loads and send data to a data collection server. Here is a sam- ple of a complete JavaScript tag that is used by a new web analytics vendor called Crazy Egg:

//<![CDATA[

document.write(‘<scr’+’ipt

src=“http://crazyegg.com/pages/scripts/1111.js?’+

(new Date()).getTime()+’“ ~CAtype=“text/javascript”></scr’+’ipt>‘);

//]]>

</script>

JavaScript log files were easier to maintain than web server log files. They also shifted the responsibility of collecting and processing data from internal company IT departments to web analytics vendors in most cases. This made implementing web ana- lytics easier. JavaScript tagging also made it simpler to innovate, to capture new pieces of data, and to do things such as set cookies to track visitor activity. Now the vendor could do this rather than having to go through the company IT department.

Perhaps the next evolutionary step in website analytics was the introduction of the site overlay (sometimes called click density). Now rather than combing through a complex set of data or pouring over tables full of data, decision makers could simply open the web page that they wanted analyzed in a browser—and for the chosen time period, the browser / web analytics application would display exactly where the web- site visitors clicked.

This democratized to a great extent what had previously been the domain of just the web analysts. It brought about increased usage of analytics solutions because now

N o t e : JavaScript tags have their own set of challenges, which are discussed in great detail in Chapter 2,

“Data Collection—Importance and Options.”

30652c01.qxd 5/1/07 12:47 PM Page 4

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ A

B R

IE F

H IS

T O

R Y

O F

W E

B A

N A

L Y

T IC

anyone could, in a very simple view, understand what was happening in the website by looking at the clicks. Optimizing websites based on customer behavior became much easier.

Figure 1.3 shows how easy it was to segment out all the traffic to the site, sepa- rating only those who came from Google and how their clicks differed. This gives us a hint of what these two segments were uniquely looking for.

Figure 1.3 ClickTracks site overlay report (segmented for all visitors and those from Google)

Currently there are four big vendors: Coremetrics, Omniture, WebTrends, and WebSideStory. There are also a whole host of mid-market vendors such as Unica, indexTools, and ClickTracks, and many basic solutions such as the open source prod- ucts AWStats, Webalizer, and StatCounter.

Google had a major effect on the web analytics landscape in 2005 when it pur- chased Urchin and subsequently, in 2006, released it as a free tool under the Google Analytics moniker. Now anyone who wanted to have access to first-class web analytics could do so for free. The number of customers using Google Analytics is hard to come by, but most estimates peg that number at half a million plus customers in the first six months. It is anticipated that Microsoft will soon follow Google and introduce a free web analytics tool.

The pace of innovation in the web analytics world continues with newer and easier ways to visualize complex sets of data from site interactions. One such recent innovation is heat maps from Crazy Egg (Figure 1.4), which is in beta at the time of writing this book. A heat map illustrates the clusters of clicks on a web page and their density by using colors (the brighter the color, the more clicks around that hot spot or link).

30652c01.qxd 5/1/07 12:47 PM Page 5

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Figure 1.4 Crazy Egg heat map report

Current Landscape and Challenges

Web analytics is, metaphorically speaking, just a toddler. The toddler has grown up a little since birth and can sort of feed itself, yet there is a lot of growth and change in front of it. This proverbial toddler sits at an amazing confluence of events.

For the longest time, companies simply spent money on their websites because it was the thing to do and it was cool. In the past few years, the Web has really “grown up” as a channel for most companies, and suddenly there is a deep demand for the web channel to be held just as accountable as the other channels (phone or retail, for example). Since the boom and bust on the Web, there has been ever-increasing scrutiny, and companies are demanding that the web leadership justify investments being poured into the channel. This is an amazingly transformative experience for the channel as its leadership is looking in all places to prove results.

Even now most people think that web analytics = clickstream. Even though this is a million miles from the truth, for most practicioners clickstream data is very much the source of all web decision making. In reality, because clickstream data is just a por- tion of web data, most companies are expressing a deep level of frustration with the lack of actionable insights, even after all the investment in expensive web analytics tools over the years. There is lots of data and even more reports, but a profound and pervasive existence of this statement in the minds of decision makers: “The data is not telling me what I should do.”

At one point during the dot-com boom, there were close to 200 vendors of all shapes and sizes in the market. Since the dot-com bust, there has been a lot of vendor consolidation in the industry. Yet the web analytics ecosystem is dominated by vendors

30652c01.qxd 5/1/07 12:47 PM Page 6

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ C

U R

R E

N T

L A

N D

S C

A P

E A

N D

C H

A L

L E

N G

E S

trying to outdo each other by offering more and more features. Vendors dominate the mindset landscape; they set the analysis agenda (in reality, it is a reporting agenda).

The lack of real-world practitioners influencing strategy and direction has had a detrimental effect. Standard techniques such as customer-driven innovation (CDI) have never taken deep roots in the world of web analytics. Most progress has been driven by possibility-driven innovation (PDI)—as in, “What else is possible for us to do with the data we capture? Let’s innovate based on that.”

There is a deep lack of actual practical knowledge out there. More important, there is a lack of people and approaches that would enable web businesses to glean insights that result in action that enable strategic differentiation vs. their competitors. Universities and colleges are not teaching practical web analytics (there is only one online course, at the University of British Columbia). This—combined with too much data (just in terms of raw size of the data that the Web has the power to throw off)— has created a suboptimal scenario when it comes to providing actionable insights for companies.

Web 2.0 and its associated technologies are increasingly becoming a part of the mainstream customer experience. This change is becoming a major disruptor for most current web analytics approaches and vendors. It is even more important in the world of Web 2.0 that we accelerate the mindset shift and the strategy for implementing suc- cessful web analytics. (For example, in the world of Web 2.0, typical clickstream data means very little because the page paradigm dies a little with every new innovation. So how do you measure success?)

With the entry of Google Analytics, the market has simply exploded, because now anyone who wants to have access to data from their website can have it for free, and from a sophisticated tool to boot. Microsoft’s anticipated free web analytics tool will only expand the options that practitioners have at their disposal. But access to the tool and data, although empowering, does little to ease the problems related to figuring out what your success metrics are and how to perform web analytics correctly.

There is more data than ever available for a web analytics practitioner to tap into:

• Competitive intelligence lets you know not only what is going on at your site, but also (for a small fee) what is going on at a competitor’s website.

• Qualitative data (usability, surveys, direct observation) gives you information about the effect of the web channel on the other channels (think Customer Rela- tionship Management—CRM).

As web analytics has progressed from birth to early infancy (now), an increas- ingly wide array of complex data has been made available. In almost every web analyt- ics tool, it is now normal to see a couple hundred metrics at the click of a button.

This increasing amount of data provides an opportunity to become better at what we can analyze and act on, yet it is also a trap (think paralysis by analysis).

30652c01.qxd 5/1/07 12:47 PM Page 7

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Companies in the web space spend millions of dollars on web analytics, chasing optimization of hundreds of millions of dollars being spent on campaigns and their websites, which are in turn chasing billions of dollars of online revenue.

Yet consistently the number one challenge in surveys, CMO priorities, case stud- ies, and fix-it wish lists is the ability to measure accurately in order to make optimal decisions for those hundreds of millions of dollars companies spend. The reason this challenge persists is that most people go about solving it wrong.

Traditional Web Analytics Is Dead

In a podcast with Internet Marketing Voodoo in March 2006 (included on the CD that comes with this book), I made the proclamation that traditional web analytics was dead. This announcement was probably two years too late.

Web analytics started its life with data sourced from web server logs, which pri- marily contain technical information and not business information. Because of this unique evolutionary path, the current crop of web analytics tools and the mindsets of customers are rooted in clickstream analysis. Figure 1.5 shows what web analytics has typically looked like.

Figure 1.5 Lots and lots of Metrics / Key Performance Indicators at the click of a button

Implementation of a web analytics tool takes just a few minutes, and instantly we have access to massive amounts of data, metrics, key performance indicators, stuff. There are practitioners and vendors and a well-established system of working and thinking in order to report on this data.

30652c01.qxd 5/1/07 12:47 PM Page 8

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ T

R A

D IT

IO N

A L

W E

B A

N A

L Y

T IC

S IS

D E

A D

So what does this world of traditional web analytics look like? If you measure any of the following sample metrics, it is likely that you live in that traditional world:

Page Views If you run an e-commerce website (or most other types), is it good or bad to have more page views per visitor? If you have frustrating navigation, you’ll have lots of page views—but no one will buy. If you have fantastic navigation, you’ll have fewer page views—but maybe people decide faster that you don’t have competitive prices and they leave anyway. Just from reporting page views, how do you know which is the case? Besides, if you track page views, what kind of behavior is being rewarded?

Hits In the early days, hits tracked the requests that a server received to send data back. Then it was okay to translate a hit as a page or content request. So more hits meant more content consumption, and it sort of meant more visitors in the very early days. Now hits means little, because of all the images and media embedded in a page. A typical page will cause 25 hits on a server. So if you are tracking hits, what are you really tracking? Requests for data to the server? Number of pages viewed? Number of visitors to the website?

Top Exit Pages If you track the pages where more website visitors exit from the site, what does it tell you? That the pages are suboptimal? It could be that they are perfect pages where your customers find exactly what they are looking for and then leave. Consider me researching a Sony digital camera on Amazon.com. I find what I want, customer reviews, and I leave. So do 99 percent of the people representing the traffic to that page. The exit rate doesn’t tell you whether your content is good or bad.

Website Engagement Ditto for engagement, often computed as sessions divided by unique visitors. If lots of people come again and again and have lots of sessions with your website, is it because they repeatedly can’t find what they are looking for or because you have the most beautiful site in the world with perfect content?

Visitor Screen Resolution Visitor screen resolution is a perfect example of a distracting metric that adds little value in any scenario. Every web analytics tool reports the moni- tor screen resolution of the website visitor and we have it in our daily reports, yet the metric rarely changes more than once every six months. Yet we keep reporting on it all the time, causing both a distraction and a sub-optimal use of time. Besides, would it not be a better strategy to simply use research from Forrester Research or Gartner on the latest trends in your industry segment and use that to drive what the size of your web pages should be?

The common theme in all these metrics is that they purport to say something, yet they say very little. Worse still, usually they actively lead us down the wrong path. At the end of spending lots of dollars to buy tools and more dollars to get reports, companies have little to show in terms of return on investment (ROI) or improved customer experiences on their websites. Years of being frustrated by an inability to

30652c01.qxd 5/1/07 12:47 PM Page 9

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

fundamentally understand the data and take action has resulted in the death of the world of traditional web analytics. We have not been able to solve for either the com- panies or their customers because after all the efforts, we have a fundamental inability to take action.

What Web Analytics Should Be

We are in the midst of a metamorphosis for our industry; web analytics is not what it used to be. This new world of actionable web analytics is about more than simply clickstream data. It also now includes data for all sorts of outcomes that can sometimes be captured by our JavaScript tags and at other times requires us to be creative about measuring. It also now includes qualitative behavior analysis: why do our visitors do the things that they do, and what is their motivation for engaging with our websites?

This expansion of web analytics means that we have a significantly enhanced ability to listen to our website customers. We have more-relevant data on tap to ana- lyze so that we can truly understand what action to take and can accelerate the ability of the web channel to be a tour de force in companies where it typically has not been.

The cornerstone of traditional web analytics for the longest time has been prepackaged key performance indicators (KPIs). But because globally defined KPIs often can’t accommodate for strategic differences in business operations and execution, they have not been quite as helpful as one might have hoped for. To compete, we have to now use key insights analysis (KIA).

Here are a few metrics that define the new world of actionable web analytics, and in turn examples of KIA:

Click Density Analysis Click density analysis, using the site overlay feature of your web analytics tool, helps you walk in the shoes of your customers. It helps you see your website as your customer does. Are they clicking on what I want them to click on? If not, what do they find interesting vs. what we are imposing on them? What do they find interesting that we were totally clueless about?

If you segment your traffic, you can see what the difference in behavior is for different kinds of traffic to your website (in this case, the clicks by everyone who comes from Google are different from those of other visitors, allowing you to perhaps target con- tent better if someone comes to your website from Google). This analysis is greatly empowering because it enables you to take action. It is not about reports and Microsoft Office Excel spreadsheets; it is about looking, literally, at your actual web pages and seeing what various segments of your customers are doing. You could take this data and begin to create customized (personalized) content for different segments

30652c01.qxd 5/1/07 12:47 PM Page 10

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

of your website visitors (hence increasing customer engagement and hopefully also moving the dial on your core success metrics).

Visitor Primary Purpose Rather than relying on the pages viewed to infer why people come to your website, in our new and improved world we simply ask customers to help us understand why they come to our website. The danger in using pages viewed to understand the visitor’s primary purpose is that if they are coming for content you don’t have, then you have no way of knowing. So why not simply ask? Conduct a sur- vey, do phone interviews. Seek out real customers and ask them why they show up on your website. Be prepared to be absolutely surprised to learn that people come for rea- sons you never imagined (reasons your business should have been accommodating for during all the years you have existed).

Task Completion Rates We are also migrating away from using clickstream data (pres- ence of a page view) to measure successful task completion. Let’s say you have a sup- port website that contains a knowledge base, answers to frequently asked questions (FAQs), and so forth. We measured success in the old world by using our clickstream analysis tool to count anyone who saw a knowledge base article or anyone who viewed a FAQ page. But does the fact that someone saw your long, complex reply really mean success? Success is extremely hard to attribute based on a page view, except in rare cases (for example, on an e-commerce website, where the Thank You page viewed after submitting an order can be counted as a successful completion of a task). In our new world, we expand our web analytics data set to include more-sophisticated qualitative data that enables us to understand whether customers can complete their tasks and whether they found what they were looking for. You can take action because there is no longer a doubt about whether a page view meant success; now you simply ask (by running survey, or doing lab usability, or creating tests on your website) and you find out and take action.

30652c01.qxd 5/1/07 12:47 PM Page 11

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Segmented Visitor Trends Few tools in the market at the moment have a real capabil- ity to segment data after it has been captured. In the old world, we embedded attrib- utes in our JavaScript tags. In our new existence, we have tools from vendors such as ClickTracks and Visual Sciences (at very different price points) that allow real segmen- tation of our data so that we don’t have to report Average Time on Site or Top Search Keywords or Popular Content for all visitors to the website in one ugly glob. Tools such as these allow you to segment your customers and their behavior in a meaningful way that allows for a significantly richer understanding of their interaction with your website. This in turn provides insights that fuel action.

Multichannel Impact Analysis The traditional world of web analytics also suffered sig- nificantly because it was based on a silo (clickstream data from your website). Yet very few companies, big or small, have their web strategy and business execution in a silo. To understand the holistic impact of the web channel, increasingly we are having to view the Web as a part of the larger ecosystem. Obtaining true insights requires meas- uring the impact of other channels (say your television or newspaper ads) on your web- site and measuring the impact of your website on other channels (how many people use your website but buy your product via retail or via your phone channel).

This extension of the worldview means that data goes out of the web analytics tool to help facilitate other types of company analysis (think of lifetime value analysis for cus- tomers acquired across all channels). It also improves the quality of our analysis by importing key relevant data into the web analytics tool (think of core company meta- data that is missing from your clickstream data, survey data, or data about offline conversions).

In a nutshell, you know you live in the world of key insights analysis when you realize that every piece of data you look at drives action—and not just action, but action that adds to whatever bottom-line outcomes that our companies are trying to achieve for our customers. (Note that important difference: not outcomes that your boss wants, not outcomes that his/her boss wants, but outcomes that your cus- tomers want.)

The world of web insights takes time to move into but after you get comfortable in it, you will have achieved a long-term strategic advantage (and a fairly substantial bonus or promotion, or both, for yourself).

Measuring Both the What and the Why

Imagine walking into and out of a supermarket. If you did not purchase anything, the supermarket managers probably didn’t even know you were there. If you purchased something, the supermarket knows something was sold (they know a bit more if you use a supermarket membership card).

30652c01.qxd 5/1/07 12:47 PM Page 12

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

Visiting a website is a radically different proposition if you look from the lens of data collection. During your visit to a website, you leave behind a significant amount of data, whether you buy something or not.

The website knows every “aisle” you walked down, everything you touched, how long you stayed reading each “label,” everything you put in your cart and then discarded, and lots and lots more. If you do end up buying, the site manager knows where you live, where you came to the website from, which promotion you are responding to, how many times you have bought before, and so on. If you simply vis- ited and left the website, it still knows everything you did and in the exact order you did it.

Hopefully you’ll see how massively advantaged the web is in terms of its ability to collect data and know lots of things about its visitors. All this without ever violating the core customer privacy principles (so, for example, most websites won’t know it was Avinash Kaushik visiting; all they know is that it was cookie ID 159ar87te384ae8137). Add to this that now there are more tools than you’ll ever realize can that will instantly create reports of all this web data, presenting it in every conceivable slice, graph, table, pivot, or dump, and you can imagine the challenge.

But, no matter what tool you use, the best that all this data will help you under- stand is what happened. It cannot, no matter how much you torture the data, tell you why something happened.

We have clicks, we have pages, we have time on site, we have paths, we have promotions, we have abandonment rates, and more. It is important to realize that we are missing a critical facet to all these pieces of data: Why did they click where they clicked? Why did our visitors end up on these pages and not those? Why do 50 percent of them abandon their carts? Why is it that 90 percent of the site traffic reads the top support questions but they still call us on the phone? What’s missing is the why.

This is the reason qualitative data is so important. It can get us substantially closer to understanding the why. It is the difference between 99 percent of the website analysis that is done yet yields very few insights, and the 1 percent that provides a win- dow into the mind of a customer.

Combining the what (quantitative) with the why (qualitative) can be exponen- tially powerful. It is also critical to our ability to take all our clickstream data and truly analyze it, to find the insights that drive meaningful website changes that will improve our customers’ experiences.

There are many types of qualitative (Why) data at your disposal, including the following:

• Brand buzz and opinion tracking

• Customer satisfaction

30652c01.qxd 5/1/07 12:47 PM Page 13

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

• Net promoter indices

• Open-ended voice-of-customer analysis

• Visitor engagement

• Stickiness

• Blog-pulse

Some of the data elements listed here cover customer interactions at your web- site, others measure what customers are saying and doing at places other than your website, and yet others measure the soft facets such as brand.

Although there are many options for qualitative analysis, perhaps the most important qualitative data point is how customers/visitors interact with your web presence.

In your quest for key insights analysis, your first stop should be understanding all you can about customer interactions at your website. A robust understanding of vis- itor interactions can lead to actionable insights faster while having a richer impact on your decision making. There is a lot of buzz around “buzzy” metrics such as brand value/impact and blog-pulse, to name a couple. These buzzy metrics can be a second or third stop on our journey because focusing on these metrics can be a suboptimal use of time and resources if we don’t first have a hard-core understanding of customer satis- faction and task completion on our websites.

There are many methodologies used to collect customer qualitative (Why) data, including the following:

• Lab usability testing (inviting participants to complete tasks, guided or unguided)

• Site visits, also called follow-me-homes (observing in a customer’s native envi- ronment)

• Experimentation/testing (the latest new and cool thing to do, A/B or multivariate testing)

• Unstructured remote conversations (engaging with your real customers remotely when they are interacting with your website, by using solutions such as Ethnio)

• Surveying (the granddaddy of them all—see the discussion of Visitor Primary Purpose in the preceding section)

If you are new to this world, the last one is a great way to get your feet wet. Unlike what you might have heard, surveying is easy to implement, can be a continu- ous methodology, is highly quantitative, and is most often chock full of insights that will lend themselves to be very action oriented.

Combining the why (intent, motivation, and so forth) with the what (clicks, visitor counts) has to be the cornerstone of any successful actionable web analytics program.

30652c01.qxd 5/1/07 12:47 PM Page 14

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

Trinity: A Mindset and a Strategic Approach

A couple of years ago while grappling with all the challenges of web analytics and how to solve them, a need arose for a new paradigm, a different framework about how to think of web analytics. Having lots and lots of KPIs, many reports full of data, and horsepower expended against all that did not yield quite the results that were expected.

Every website had a standard big-three web analytics package installed for a few years, reports were published, and victory was declared upon successful completion of the nightly reports. But if business as normal did not yield any insights to improve the customer experience on the website, then what should the paradigm be?

The answer was Trinity, a new way of thinking about decision making on the Web—something that was more than clickstream. Trinity is also a framework that can empower your web strategy. Executing to the Trinity will ensure that you can build a world-class decision-making platform that will create a sustainable competitive advan- tage for your company.

The central raison d’être of the Trinity strategy is something radical: actionable insights and metrics (see Figure 1.6).

Figure 1.6 Solving for actionable insights and metrics

The goal of this strategy is not to do reporting. The goal is not to figure out how to spam decision makers with reports full of data via email. Actionable insights and metrics are the über-goal simply because they drive strategic differentiation and a sus- tainable competitive advantage.

Having actionable insights combined with clear goals helps crystalize the efforts of the organization. If you are doing things (reports, data analysis, meetings, reviews, and so forth) that are not singularly geared toward producing actionable insights, then stop. This strategy encourages the right behavior from the organization and is a great way for key stakeholders to make their day-to-day resource allocation decisions.

Behavior Analysis

The first component of the Trinity mindset is behavior analysis, what we traditionally consider clickstream data analysis (Figure 1.7).

Actionable Insights &

Metrics

30652c01.qxd 5/1/07 12:47 PM Page 15

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Figure 1.7 Behavior analysis, attempting to infer customer intent

The goal of behavior analysis is to (as best as we can from the data we have) infer the intent of our customers or website visitors based on all that we know about them. We will not follow the rest of the crowd and expect too much of the clickstream data. The best we can do with clickstream data is infer intent, and we have to make peace with that.

After collecting clickstream data, the objective is to analyze it from a higher plane of reference. No more measuring hits or overall time on site or visitor counts or top exit pages. Under the Trinity strategy, we will do click density analysis by using the site overlay report. We will massively segment the data by n levels to find core nuggets of valuable insights. We will do search analysis (and not just external keywords, but also internal site searches). The objective is to become really smart about clickstream analysis and to start truly inferring the intent of our site visitors.

There is a downside to inferring intent: two people may look at the same set of data and clicks on the website and form differing sets of interpretation. This is usually because each of us is a collection of our own unique background and experiences. The great thing about acknowledging that we are inferring intents is that we are free to make those inferences, present them to our peer group, validate them, and then draw conclusions and make recommendations.

Outcomes Analysis

The second component of the Trinity mindset is outcomes analysis (Figure 1.8). I fondly call it the so what element.

This is critical for one simple reason: at the end of the day when all is said and done, you want to know the outcome for the customer and the company. This element also solves one of the critical flaws of traditional web analytics, that of an egregious amount of focus on page, time, and visitors metrics derived from clickstream data. Because web analytics has its roots in log file analysis (which never had outcomes), for the longest time it had lots of data and metrics but not the most important one—an answer to, “So what happened, what was the outcome?”

Actionable Insights &

Metrics

Behavior

ClickStream Click Density Analysis

Segmentation Key Metrics, Search

Intent Inferences

30652c01.qxd 5/1/07 12:47 PM Page 16

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

Figure 1.8 Second Trinity element: outcomes analysis (so what)

I encourage you to ask a simple question to the site owners: Why does your website exist? You might be surprised how many can’t answer that question in 15 words or fewer. This element of the Trinity exists to measure how well the website is doing in meeting the goal of its existence.

In the simplest of terms, this measures revenue for e-commerce websites (not just how much, but also why we made as much as we did) and measures conversion rates better. But for support websites, this measures problem resolution and timeliness. For websites that exist for lead generation, this element of the Trinity measures the number of leads and the quality of those leads (and whether the quality improves over time). For your website/business, the outcomes could be different from the ones listed in the illustration, but they will almost always be metrics that affect the bottom line and can be tied to the financials of the company.

Every website should have a clearly articulated outcome. If you don’t have the capacity to measure all nuances of outcomes, the recommendation is to give up on measuring behavior (clickstream) altogether. If you don’t have the ability to measure outcomes robustly, all the behavior analysis in the world will do you no good because you will have no idea whether all those graphs in the web analytics application you are using that are going up and to the right added any value to your company. Is it a bit extreme to dump clickstream in favor of measuring outcomes first? Yes. Necessary? You bet.

Actionable Insights &

Metrics

Behavior

Outcomes

ClickStream Click Density Analysis

Segmentation Key Metrics, Search

Intent Inferences

Orders/Leads Revenue: How, Why Conversion Rates

Problem Resolution Nuances of Outcome

30652c01.qxd 5/1/07 12:47 PM Page 17

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

Experience Analysis

The third and last component of the Trinity mindset is experience (Figure 1.9). This encompasses our dear elusive best friend, why.

Figure 1.9 Third Trinity element: experience analysis (the why)

Although the outcomes element of the Trinity is mandatory, the experience ele- ment is perhaps the most critical. For any organization that is stuck in a rut and unable to find any real actionable insights from behavior and outcomes—no matter how hard they try—the recommendation is to invest in experience analysis. This is the why. This is the warm hug when you are stymied and tortured by your clickstream data and you want to tear your hair out.

It is hard to choose a favorite among your children, but for me experience is without question the favorite. The reason is quite simple: experience analysis allows us to get into the heads of our customers and gain insight or an a-ha about why they do the things they do.

There are many ways to understand the experience of customers on your web- site. You can leverage surveys and simply ask them, or there are very complex statisti- cal quantitative and qualitative methodologies you can bring to bear. Surveys will allow you to measure customer satisfaction and even predict future behavior (likelihood to buy or to recommend your products or services). As will be clear throughout this book, I am a huge believer of experimentation and testing (let’s have the customers tell us what they prefer) by using either the A/B testing methodology or multivariate testing. We also have the traditional user-centric design techniques at our disposal, such as

Actionable Insights &

Metrics

Behavior

Outcomes

ClickStream Click Density Analysis

Segmentation Key Metrics, Search

Intent Inferences

Orders/Leads Revenue: How, Why Conversion Rates

Problem Resolution Nuances of Outcome

Experience Research Customer Satisfaction

A/B Testing Heuristic Evaluations Voice of Customer

30652c01.qxd 5/1/07 12:47 PM Page 18

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

heuristic evaluations. We can also leverage lab usability testing as another great option or do follow-me-homes (site visits), a concept advocated by Scott Cook, the founder of Intuit as the essence of the Customer Driven Innovation (CDI) mindset.

All these experience methodologies are solving for one single purpose: getting companies to listen to the voice of the customer, a voice that in most companies and corporations is lost in the wilderness.

Solving for Companies and Customers: Win-Win

In the end, the Trinity mindset drives the fundamental understanding of the customer experience so that you can influence the optimal customer behavior that will lead to win-win outcomes for your company and your customers (Figure 1.10).

Figure 1.10 Understanding experience to influence behavior for win-win outcomes

That last part is important: Trinity aims for win-win outcomes. If the right version of the product for a particular website customer is Basic and

not the Professional edition, our job as site owners is for us to help customers figure that out and buy Basic. Yes, we can make more money in the short term if the cus- tomer buys Professional today. But it is quite likely that the customer will buy Profes-

Actionable Insights &

Metrics

To influence optimal Behavior

Leading to win-win Outcomes

Understanding explicitly Customer

Experience

Behavior

Outcomes

ClickStream Click Density Analysis

Segmentation Key Metrics, Search

Intent Inferences

Orders/Leads Revenue: How, Why Conversion Rates

Problem Resolution Nuances of Outcome

Experience Research Customer Satisfaction

A/B Testing Heuristic Evaluations Voice of Customer

N o t e : Chapter 3,“Overview of Qualitative Analysis,” covers all of these user-centric design methodologies

in greater detail.

30652c01.qxd 5/1/07 12:47 PM Page 19

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

C H

A P

T E

R 1

: W

E B

A N

A L

Y T

IC S

— P

R E

S E

N T

A N

D F

U T

U R

E ■

sional, use it, get frustrated because it is too advanced for them, and we’ll never see them again (and they will share their suboptimal experience with others). But if we help them buy the right version, Basic, then next year they’ll be back for Professional. Trinity aims to solve for the long term.

Understand the needs and wants of your customers and solve for that. By using the Trinity, you can and will win big—and frankly it is a blast solving the problems of your customers when you know what they are.

Each element of the Trinity is supported by a different tool. The Trinity incorpo- rates different methodologies while leveraging repeatable processes. Most important, it requires key people skills. Just having the mindset does not solve the problem (though it will put you on the right path). Executing the Trinity strategic approach means creat- ing the right organizational structure and an evolved culture.

Building an Integrated Trinity Platform

The entire framework will not come into existence overnight. Typically, you’ll diagnose what you currently have and will work toward putting the missing pieces of the puzzle together. It is important to ensure that your execution strategy plans for, and postim- plementation allows, your analysts to have the ability to tie all the elements of the Trin- ity together (Figure 1.11). This will be a massive advantage for your company.

As an example, if your visitors place orders or submit leads on your website, some anonymous tracking elements such as the transient session_id and cookie_id can be passed to the orders database. This will allow you to do deep segmented analysis of outcomes and of the behaviors that drive those outcomes.

Another example is passing the (again anonymous) session_ids to your survey tool so that you can segment the most unhappy customers based on survey results. You can then use your web analytics tool to analyze the clickstream data and see what pages happy customers saw as compared to unhappy customers. Or you could see what website experience drives better customer satisfaction, and so forth.

In a world where your analytical world view was severely limited by using only your web analytics tool and clickstream data, the Trinity framework expands the available data, helps you truly understand your customers, and allows you to slice and dice your data to gain holistic insights. Satisfaction of your customers and revenue for your company soon follow. (And your analysts are happy to boot because they finally have the challenge of analyzing qualitative and quantitative pan-session data— what fun!)

30652c01.qxd 5/1/07 12:47 PM Page 20

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

■ W

H A

T W

E B

A N

A L

Y T

IC S

S H

O U

L D

B E

Figure 1.11 Integrated Trinity strategy (primary keys allow key silos to be tied together)

The Trinity framework can be applied to any kind of web business: e-commerce, support, advocacy, nonprofit, small business, and so on. You will find more details and specifically how to apply Trinity to your business in Chapter 6, “Customized Solutions to Jumpstart your Web Data Analysis.”

Experience

Behavior

Clickstream Clik Density Analysis

Segmentation Key Metrics, Search Intent Inferences

shopper_id session_id test_value

site_id

Research Customer Satisfaction

A/B Testing Heuristic Evaluations Voice of Customer

shopper_id session_id

Outcomes

Orders/Leads Revenue: How/Why Conversion Rates

Problem Resolution Nuances of Outcomes

shopper_id session_id

tracking_code

30652c01.qxd 5/1/07 12:47 PM Page 21

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

30652c01.qxd 5/1/07 12:47 PM Page 22

C o p yr

ig h t ©

2 0 0 7 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .