What comes from Google Analytics, Adobe Site Catalyst, WebTrends, CoreInsight / NetMetrics and more.

Why comes from lab usability studies, website surveys, "follow me home" exercises, experimentation & testing, and other such delightful endeavors.

Why gives context to the What, and delightfully helps you not have to overlay your biases when you try to infer visitor intent form all the What (clickstream) data.

I know that you agree Why is important.

I know that you even realize Why is ever easier to accomplish (usability studies are economical, surveys and testing platforms start at the sweet price of free!).

Yet your site stinks like a skunk.

The reasons are complicated.

You are smart, so that is not it. Maybe it is internal politics. Maybe it is the agency you have outsourced the site to, the agency whose only competence seems to be gratuitous use of flash. Maybe it is that it is not your job, you are the "quant" guy or "GA girl". Maybe even after taking one of the team and going out on three dates the IT Dude still refuses to put Website Optimizer tags on the site. Maybe the well meaning but "never met our real customers" HiPPO dictates site design.

Bottom-line: Your site stinks and you need to fix it.

Allow me to introduce you a User Centric Design that is, I think, the solution you have been waiting for: Heuristic Evaluations

I love heuristic evaluations because they are cheap, fast and you probably already have resources you need in your company. A large part of my adoration also comes from the fact that heuristic evaluations are us going back to the basics in an attempt to create un-stinky websites.

What Are Heuristic Evaluations?

A heuristic is a rule of thumb. In as much, heuristic evaluations follow a set of well established rules (best practices) in web design and how website visitors experience websites and interact with them.

When conducting a heuristic evaluation a user researcher (or an HCI expert) acts as a website customer and attempts to complete a set of predetermined tasks related to a website's existence. For example: Trying to place and order, or looking to find out the status of an order, or the solution to an error code, or decide which of many products on a site are optimal for a specific customer persona.

But here is the lovely part, and why almost anyone can perform heuristic evaluations. in addition to best practices the researcher (or you!) will raw from their own experience, knowledge and common sense.

Heuristic evaluations are best when they are used to identify what parts of the customer experience are most broken on your website. They can be very beneficial if you have not yet conducted any usability tests or when you would like to have a quick review of prototypes that the designers might be considering.

In either case, you can quickly determine the lowest hanging fruits in terms of "broken" parts of the customer experience. With this feedback there can be iterative improvements to the customer experience. You'll probably already have connected the dots and realized that this is a fantastic way to identify ideas for A/B or Multivariate experiments on the live website.

[I Heart] Group Heuristic Evaluations

There is one more thing, a way to amplify the impact and get even better results.

Get everyone involved!

Heuristic evaluations can also be done in groups!!

Invite people around you with key skills, such a designers, information architects, web analytics professionals, that girl in accounting you really like, other analysts in the company (and their quantitative understanding of site data), search experts, the intern who only communicates via Posterous, and so on and so forth.

The goal is simple: Identify flaws by attempting to mimic the customer experience (if possible under the stewardship of a User Researcher, if not then under the gaze of your haunting brown eyes) by completing the tasks on the website as a custom.

The great benefit of the group heuristic evaluation method is that you can tap into the "wisdom of crowds". This is especially powerful because the web is such an intensely personal medium, and the group members can offer different points of view that highlight issues quickly.

The process that worked optimally for me was to send an email to 10 or so folks (a diverse set!). Invite them to a noon meeting in a largish conference room and order lunch for them (best $50 I ever spent). Once everyone was settled (by 1205!) project the website on the screen and try to complete the most common customer tasks.

I (or you) have to do a good job of moderating the discussion and ensure everyone participates. There is no such thing as a bad opinion, diversity is good. Collect all pertinent feedback.

Heuristic evaluations can provide valuable feedback at a low cost ($50 in my case) in a very short amount of time (an hour in my case) and identify obvious usability problems. Hence they are best for optimizing work flows, improving user interface design and for understanding the overall stinkiness (or lack thereof) of the website.

Here is another subtle benefit of the group evaluations: improved communication and, dare I say, camaraderie between different groups in your company (big or small).

There is nothing that quite brings people together like a,  pardon the expression, bitchfest. Everyone contributes, everyone commiserates, everyone loves it. Next time they are doing something they'll know what you do. Next time you need help, you'll know who to call (and they'll pick up the phone!). It is a great way to bring a sense of common purpose and a sense of ownership to improving the website experience.

Conducting A Heuristic Evaluation (The Glorious Process!).

You're excited right?

Here are six steps to conducting a successful evaluation process, either when you do it or you are doing it as a group:

1. Write down the tasks that customers are expected to complete on the website. If you are using surveys on your site (even a simple site level survey like 4Q from iPerceptions or page level survey like Kissinsights) then that is a fantastic source of this information. You should also, if possible, talk to the site owner. Here is what you might end up with on your list:

• Find information about the top-selling product on the website.

• Locate a store closest to where the customer lives.

• Place an order on the website by using PayPal. (If the website doesn't accept PayPal, how easily and quickly can a customer find that out?)

• Sign up to show up at a protest march against taxes on the richest Americans.

• Successfully contact tech support via email.

• Pick the right product for customer profile x (where x can be a small business owner or a family of four or someone who is allergic to peanuts).

2. This is sometimes hard but try to establish success benchmarks for each task. For example success rate for placing an order for the best selling product = 80%, signing up for the protest march = 99%, contact tech support = 90%, etc etc.

3. The fun part. Walk through each task as a customer would and make note of the key findings from the experience – everything from how long it takes to complete the task to how many steps it takes to the hurdles in accomplishing the tasks to how profound your embarrassment was that this was your own company's website.

4. If you were using a best practices checklist (more on this below) then make a note of the specific rule violations.

5. The hard part. Create a report of the findings. You can use PowerPoint with a screenshot of the webpage and clear call-outs for issues found. Or you can use Camtesia / a screen recording software to capture the session (and the group discussion). This can be distilled to a "the best of the bitchfest" collection for your superiors.

6. The hardest part. Categorize the recommendations into Urgent, Important and Nice to Have. We all get swept into emotions fervor. It is also possible that when you present your findings to your Sr. Management they might be a bit HiPPOish (not that there's anything wrong with that).

You want to go in with the Urgent, Important and Nice to Have based on impact on the customer experience and the company bottom-line. This helps drive a "what we should do" discussion rather than "I think we should do that" discussion.

That's it.

The process, as always in web analytics, is important and I hope the above six steps help you create a process in your company that is repeatable and yields impactful results.

I want to stress again that this is a great way for you to get into the customer's shoes, for your to build camaraderie, involve the cross-functional team of people, and finally find the lowest hanging fruit for sure and perhaps even some big ones.

The Cheapest Heuristic Evaluation Exercise.

I have hinted about the cheapest possible heuristic evaluation exercise a couple of times in this post.

It is: You sitting down with your common sense and a list of "best practices" and checking how well, or badly, your website does against that list. Then do steps 5 & 6 above.

This is fast and impactful. Even in the worst case you identify the most broken things / annoyances.

I have used lots of website best practices usability checklists over time and have settled on using Dr. Peter Meyers's 25-point checklist. It is simple, effective and quite expansive. You can download it here. [If you' have Web Analytics 2.0 then you'll also find an "extended edition" on the CD that is attached to the back cover of the book.]

The usability checklist has four sections. Here's a brief summary:

Accessibility

1. Site Load-time Is Reasonable
2. Adequate Text-to-Background Contrast
3. Font Size/Spacing Is Easy to Read
4. Flash & Add-ons Are Used Sparingly
5. Images Have Appropriate ALT Tags
6. Site Has Custom Not-found/404 Page

Identity

7. Company Logo Is Prominently Placed
8. Tagline Makes Company's Purpose Clear
9. Home-page Is Digestible In 5 Seconds
10. Clear Path to Company Information
11. Clear Path to Contact Information

Navigation

12. Main Navigation Is Easily Identifiable
13. Navigation Labels Are Clear & Concise
14. Number of Buttons/Links Is Reasonable
15. Company Logo Is Linked to Home-page
16. Links Are Consistent & Easy to Identify
17. Site Search Is Easy to Access

Content

18. Major Headings Are Clear & Descriptive
19. Critical Content Is Above The Fold
20. Styles & Colors Are Consistent
21. Emphasis (bold, etc.) Is Used Sparingly
22. Ads & Pop-ups Are Unobtrusive
23. Main Copy Is Concise & Explanatory
24. URLs Are Meaningful & User-friendly
25. HTML Page Titles Are Explanatory

Seems simple right?

I bet your website currently breaks 10 of the rules above. It is hard to believe. Set up a quite hour aside. Go through the checklist. But first go download the detailed checklist at Dr. Pete's website.

When you are done remember to do steps 5 and 6 of the recommended heuristic evaluation process outlined above.

Benefits of Heuristic Evaluations.

In case you somehow made it here and were not convinced of the value doing heuristic evaluations here is a quick summary of the benefits:

• Heuristic evaluations are extremely fast to perform, with a very short time to insights.

• They leverage existing resources in your company (what could be awesomer?).

• You'll identify the most egregious customer issues on your website (often all the low and medium hanging fruit, quickly).

• They can be used very effectively early in the website development process to find potential customer hurdles / deal breakers.

• If you have an existing UCD program or hire an external company/agency. heuristic evaluations can reduce the cost of full usability tests by helping fix the obvious problems. The $900 an hour charged by the Agency can then be focused on hidden / really tough challenges.

Thing to Watch For.

It should be clear that I am a fan of the heuristic evaluation process. And it is every fan's duty to also highlight things to watch out for. Here they are:

• Both single person led or group evaluations contain company employees, and sometimes usability experts, but none of them are the actual customers. Despite using best practices and our wisdom we might miss some subtle problems, even as we identify the obvious ones. Be very aware of this.

• [It follows from the above point that.]  The better you are at step #1 outlined in the heuristic evaluation process, the better your outcomes will be.

• When there is disagreement in recommendations from the heuristic evaluations there can be great value by doing live website tests or usability studies (whatever is faster, usually experiments).

• Heuristic evaluations are best for optimizing work flows, larger more obvious parts of website design and the overall usability of the website.

In summary: Not quite God's gift to humanity, but rather the best thing you could do to identify the low and medium fixes to your site that will significantly improve the experience of your customers.

Don't spend your day immersed in Google Analytics and just the "What" analysis. Understanding "Why" is the key, use it to unlock actionable insights.

Ok your turn now.

Have you done heuristic evaluations for your website? Who leads them in your company? Do you have a list of usability best practices that you use on your website? What other methods of of listening / collecting voice of customer / answering "Why" do you use in your company?

Please share your feedback / critique / questions / answers via comments.

Thanks.

PS:
Couple other related posts you might find interesting:

• Overview & Importance of Qualitative Metrics
• Eight Tips For Choosing An Online Survey Provider
• Measuring Success for a Support Website: A Point of View
• Is Conversion Rate Enough? It's A Good Start, Now Do More!
• Multiplicity: Succeed Awesomely At Web Analytics 2.0!

Qualitative Web Analytics: Heuristic Evaluations Rock! is a post from: Occam's Razor by Avinash Kaushik

The hard nut is not that we all are doing basic reporting about Visits and Bounces. Ok doing just that is lame. But still that's not all of it.

It might shock you that the hard nut is not even that we, the people who "play" with data, the Analysis Ninjas if you will, are not leveraging custom reports and advanced segmentation and activity alerts and all that cool stuff.

It is that no one wants to take action on our data. Ok not no one. But most of the time no one takes action based on our hard work.

Why?

To illustrate perhaps the most obvious, and yet hidden, problem I wanted to share with you a recent experience I had with data. It was not in the context of web analytics and yet I think it presents a solution to our problem.

On my (beloved) Nexus One android phone one of my absolute favorite applications is Cardio Trainer. I just love it. The UI and the UX and how it is so darn easy to use (and much more well thought than competitors like My Tracks). You must get it.

I use it when I do any kind of exercise (or hike with the kids). You just open the app, say Start Workout, choose the type of work out (running, biking, elliptical, horseback riding, and 15 other choices) and you are up and running.

It starts tracking you (GPS and what not), pauses when you pause, and if you choose even talks to you (motivation!).

When the workout is done it gives you. . . . data!

Awesome right? What analyst / data person would not love this.

There is a sweet map (I can switch to the satellite view and see the buildings I went around!) and there is lots of data.

Time: 0:22:07  Distance:  6.74 kilometers Speed: 18.3 km/hr Calories: 207

It is a great dashboard.

For the first few times I used CardioTrainer I loved seeing the data (almost more than the fast bike ride). I would switch the map to the satellite view and drill down into the woods and the lake and the trees.

Of course I would look ay the data and maybe remember fragments.

Remind you of something?

Perhaps your web analytics dashboard. Or perhaps your "data summary report". Or, hopefully not, the data puke that goes to the entire management team.

Initially everyone's happy, nothing much happens, yet it feels good.

In my case pretty soon I switched to another view of the data, this one. . . .

Shorter summary. No gimmicks like the map. : ) A bit more data.

I can see Total Climb (elevation): 16 m. I can also see the min and max for Speed (max 25.1, min 7.5). Pretty good right?

This view felt better. It seemed like less data.  Bit easier to remember the Calories.

I am embarrassed to admit that having two more speed numbers (min and max) made me forget the speed even faster, and that's a number I love to remember because I want to constantly beat it.

More data did not make life any more data driven.

Then the other day I noticed that CardioTrainer was auto-updated over the air (love this Android feature!).

When I was done with the workout I was astounded to see this when I hit the End Workout button. . . .

All my beloved data was gone and it was replaced by this simple message:

2 Pears

OMG! Awesome!

Here is what was so cool (and not it was not just the Great Job! line):

1. All that data I had before had meant nothing. I remembered little. There was not much I changed or actioned based on it. Having data felt good. Did nothing. [Remind you of your Visits, Visitors, Page Views and Bounce Rate reports?]

With the new "dashboard" I realized I needed to ride more / harder instantly. Two pears was simply unacceptable!

[I am up to four pears, in just one week! ]

2. This one is key.

The CardioTrainer team took time to truly understand why I, and many others, workout and translated all that data into a clearly relatable and understandable "bottom-line outcome".

I exercise to stay fit (and not become overweight!). Calories and elevation and speed and distance are all great. But they are two steps removed from the outcome.

All those numbers don't matter. It is hard for the "HiPPO" (me!) to translate that data to understand what was the impact / outcome. Burning more calories seems obvious, but how much? And what did 206 calories mean?

Not any more. Now I can take those pears and tie them to how much I had consumed all day (vegan healthy food!), make a decision and take action.

I had all that data before, detailed data, and yet I hardly new what it meant. Now it was crystal clear: I had burned off an equivalent of 2 pears. I needed to change.

When you think of your day job as an Analyst. . . are you simply communicating data or do your reports have the equivalent of two pears?

For most of us the answer is no. No pears.

We love and provide data.

Not surprising that getting leaders to make decision is such a hard nut to crack right?

Here are two lessons I took away from this experience that are relevant to our efforts to create a data driven experience.

The "2 Pears" Web Analytics Action Driving Philosophy:

1. Simplify & focus.

Become a ruthless killing machine when it comes to metrics/data. Especially if the audience is our beloved HiPPO's or Sr. Management.

Our job is not to impress them with data or show them how clever we are with our segmentation strategy. Our job is to communicate, convince, drive action. CCDR. If you need a new acronym.

Ask yourself why every piece of data is on the report / dashboard / presentation. If it is not "instantly useful" kill it. And enjoy the kill.

Here's how: Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test

2. Truly madly deeply understand the desired outcome.

The glorious thing was not just removing the data, it was the profound tie between the activity (exercise) and the outcome (more pears!).

In our case activity = website and outcomes = what?

That is the secret magical awesome question that needs answering.

It is not page views and bounce rates and time on site and browser versions. Surprisingly it is not even conversion rates or goals met. Very often all these metrics fall in activity.

What is the point of www.generalmills.com? Videos? Downloads? NO!

What is the point of www.lorealparisusa.com? Ecommerce? Surprisingly.. NO!

What is the point of www.nytimes.com? Page views? Not really.

What is the point of www.doctorswithoutborders.org? Current disasters listing? NO!

As an analyst for each site my job is to figure out what the "2 pears" are. I mean really figure it out. Not look for proxies. Pears! Talk to people, beg your management, be willing to go out on dates and take one for the team.

What is the one thing that matters the most, the reason why everything else happens, the singular reason all that data exists?

Then focus just on the pears on my Sr. Management reporting / presentations.

What was the change in "pears"? What we should do more? What should we do less?

End of the story.

Here's how: Win With Web Metrics: Ensure A Clear Line Of Sight To Net Income!

That's it.

Two simple lessons that if applied well will begin to move you towards communicating your brilliance the way it was meant to be communicated, and shift your organization to being more data driven.

Now every single time you look at your dashboard / performance summary / reports ask yourself this simple question:

Got Pears?

Ok it's your turn now.

Is the reporting you do crystal clear on the "pears"? If not why not? Have you seen other examples in the wild of insights presentation that have made you go "ah ha!!"? Care to share tips you have used to communicate effectively with your data audience? What is your company's secret to taking action based on data?

Please share your thoughts / critique / tips / screenshots / experience via comments.

Oh and before I forget. . .

If you are on the Android platform go get CardioTrainer now!

Simply scan the QR code on the right with the Barcode Scanner app and before you say abracadabra you'll be on your way to happier data driven workouts!

PS:
Couple other related posts you might find interesting:

• Web Analytics Measurement Framework
• Who Owns Web Analytics? A Framework For Critical Thinking
• 10 Fundamental Web Analytics Truths: Embrace 'Em & Win Big
• Excellent Analytics Tip #13: Measure Macro AND Micro Conversions
• Excellent Analytics Tip#4: Make Your Analysis/Reports "Connectable"

Win Big With Web Analytics: Eliminate Data & Eschew Fake Proxies is a post from: Occam's Razor by Avinash Kaushik

My hope in this post is to share some simple tips with you that might make your reports and analysis speak to you a bit more. Suggestions that might increase the probability that you'll bump into things that might be insightful, and communicate data more effectively.

None of them are very hard to do, but I think they make a world of difference.

Excited? Here we go. . .

#1: Go as deep as you can. Then, a little bit more.

Far too often in our daily lives we let our job titles limit how deep we go in our analysis.

For example let's say I work at a delightful car / health / spaceship insurance company. Naturally all of my analysis is focused on the efficiency of the website in moving the Visitors quickly from the landing page to click on that delightful Submit Quote button.

I am focused on what the site does because that is what my job title says: Web Analyst

I am analyzing campaigns (which ones convert better and which worse), I am looking a little bit at the bounce rates, and of course I am totally obsessing about my seven step quote submission funnel (and how to reduce abandonment).

Bottom-line: Quote, quotes, quotes.

And that is fine.

The data is easily available in the web analytics tool so why not.

Here's my advice: You should kick things up a notch. Don't focus just on the quote (the part the site does), include the final conversion to a paying customer (even if that data is offline).

The picture you get from stopping at Quotes might be very different from stopping at Policies Purchased.

Here's what you are focusing on (and it is good):

All my experience in these things suggests that it is dangerous to think that the Conversions column is representative of the final outcome.

Here is what it probably looks like (and this is going from good to great):

See how the ranking changed?

You would make different recommendations right? Would it save your company money? Would it make you refocus your efforts on where improvements are needed?

You betcha!

For straight ecommerce websites the first picture is what you use every day. But for most other types of businesses the final success does not exist in web analytics tool. So what? Get the data out of the crm / erp / "backend" system. . . dump it into excel. . . write a simple formula!

Usually you don't need a complicated multi year data warehousing effort with expensive business intelligence tools to buy. At least for this scenario you just need a column and a short movie data with your online IT person and a longish coffee break with your "backend" IT person to get the right primary keys set up. Then you can bring your sexy back!

Go deep.

You are paid to find real bottom-line impacting insights (remember line of sight to net income?). Do that.

If you are a purely ecommerce business then you can go a bit deeper too. Consider doing quarterly analysis that focuses on calculating customer lifetime value. Up a notch.

If today you are a content site that is only focused on measuring content consumed try to go deeper to understanding CPA of the ads or Visitor Loyalty. Once again going one step deeper, up a notch.

And so on and so forth.

Make it a point to pause every Friday at 0900 hrs. Look at your most important work / report / dashboard. Then ask yourself this: "How can I take my view of the data one step deeper?"

Now figure out how to do that. That'll impress me, your boss and your mom.

#2: Join the PALM club. [PALM: People Against Lonely Metrics]

This rule actually comes from my second book, Web Analytics 2.0. [Page 318, Principles for Becoming an Analysis Ninja, if you have the book already.]

The rationale for this rule, joining the PALM club, is quite simple.

You need a someone in your life. I need someone. Everyone needs someone else. A boy friend. A girl friend. A cat. A "you complete me" person.

So why not your metrics?

We do reports / dashboards like this one all the time:

Ok great.

I know the top referrers sending traffic to my site in a month. Maybe I can appreciate more the power of Twitter or google.co.in or whatever.

You might even impress me next month with a updated version of this where some of these might have shifted a bit up or a bit down.

I might not do anything with the data… but you surely hypnotized me for a few seconds.

This is the problem with lonely metrics.

They don't have any context. They fail to communicate if 841 visits from Twitter were any good. In fact is any of the above good or bad? How do you know?

Why not find a BFF for your lonely metric and present something like this. . . .

Much better right?

I found a "you complete me" for my Visits metric, Bounce Rate.

Now in an instant I can not only see which referrers are big or small, I can see which ones are "good" or "bad".

I could have picked conversion rate as the bff. I could have picked % new visits. I could have picked connection speed or mobile platform or underwear size.

Whatever makes most sense for my business. But putting two minutes of thought into my metric would help make my report a little bit more useful.

Kick it up a notch. Right?

Never ever never never never ever present any metric all by itself.

If you want a cop out then at least trend it over time. If you actually want love then join PALM and don't let your metric be lonely.

Let me close with one of my favorite examples of this rule, this one's to inspire you if you have a pure content (non-ecommerce) website. . . .

Good to know what content's being consumed. Column: Pageviews.

Much much much better to know what the $ index value is for each.

See that crazy blue line that's literally off the chart? You would want to know that about the 1,414 pageviews right?

Now go find your dashboards, your reports, your data pukes (sorry!) and make sure that for every dimension you are not reporting success or failure using just one metric. Join PALM!

[Tip: Not that you are trying to but if you want to impress me but if you are then make sure the second metric you pick is as close to an outcome metric as possible. Or an actual outcome metric. I. Love. Outcomes.]

#3: Measure complete site success. Measure everyone's success.

One of my greatest passions when doing analysis is to look at the complete view of things. Rather than just the obvious.

An application of that passion is to look at all the jobs the website is doing, representing all the work that is being done by people in your company who touch the site.

Ecommerce is too easy an example of this so let me use a non profit example.

San Francisco Aids Foundation is a charity I support. It does incredible work to prevent new HIV infections.

The only way SFAF stays in business is if you and I make donations. As an Analyst I would focus all my energies on trying to figure out how many donations we are getting and where those people come from and what they are doing on the site etc.

But donations is just one measure of success ("macro conversion"). There are other jobs that the site is trying to do, and people who work on those jobs. So why not measure those?

For example. . . .

Measure the above four micro conversions, in addition to the macro conversion of donation, helps give a complete view of success. And what to do better.

Maybe Google is really good at Volunteers and not optimal for attracting people who donate. If you focus only on donations you'll devalue Google. Or maybe facebook is the best source for sharing information (downloads). And more such things.

Not only are you measuring all that matters. . . . you are validating the jobs of people who put together all that content.

Most of the time we don't do this. We, web analysts, just focus on one thing and then we wonder why we don't have the impact we want to, or why everyone does not pay attention to us.

Broaden your view!

If I were analyzing Amazon I would measure sales AND affiliate signups, signups for amazon prime, credit cards, wish lists, "like's" on reviews, self publish inquiries, free downloads….

If I were analyzing L'Oreal Paris it would be sales AND reviews, coupons downloaded, successful completion of "Profile My Skin", videos watched, sign ups for mobile alerts….

In both cases a complete view of the website and success of every person who works on the site.

Ninjas do that. You should too.

#4: Be smart about using time. Move beyond MoM.

This is one of the most common view of data presented in web analysis…

The picture illustrates the performance of a metric over two consecutive months.

This is of course better than just showing data for June.

The problem occurs when you proceed to look at six such graphs on your dashboard and then proceed to use the trends to deliver insights. You are reading too much into the ups and downs, you are inferring things that might not even exist.

Two months do not a trend make. Important lesson.

Not even for the world's most flat line no seasonality business.

So here is a best practice. . . . at least add three months. . . . if the data looks like below you'll think one thing (and every different from above pic)…

But if the data looks like the image below. . . . you'll think something else. . . .

Worry in one case. Jubilation for the temporary awesomeness for May in the other.

The more time you put into this graph (and if you have as much space as above you can easily add at least six months and it will still look pretty) the better.

But if I can only have three I love using current, prior, same month last year.

Better context right? Will take you off on a completely different line of inquiry, all from adding June 2009 to look at June 2010.

If June is the last month of your quarter and you have a cyclical business then maybe you want to compare Apr, May, June 2010 and have the first column be March 2010 because you want to see how the last month of this quarter did vs last month of the last quarter (because Apr and May don't really show if the trend ended as high or low as it should have ended).

So on and so forth.

Remember:

Sweet momma that is awesome!

Kick it up a notch, ok?

#5: Present data better, make insights obvious.

There are so many ways to present data that a small section of a blog post is insufficient. And of course there are so many people who are better at this than I am.

Let me just say that the way you present data matters, a lot. I'm not saying you should make it pretty. I could not care less if it is pretty or not. I'm saying present it in a way that the insights you think exist in the data become more obvious.

Here is a "data element", from an actual dashboard, that I really like. It might not be sexy but it is extremely functional and it is super awesome at communicating the smarts of the Analyst.

Three month trend for one very important business metric…

First note that rather than just showing one column for the performance of this metric it shows four. One for each key segment of the customer that the company has.

This would require you to know the business (good thing), know its customers (great thing) and track the segmented data. IE have your act together.

Second note that the data is for three months. You could show more but in this case you don't want to overwhelm the Executive. If you go more months, shrink the segments.

Third, really important, note that the overall goal is clearly indicated in the picture. 80. And to get that number you would have to talk to Finance and Marketing and HiPPO's and get an agreement up front. This is absolutely magnificent, key to you building relationships and finding insights.

The nice thing about our picture above is that the overall metric would get averaged out and show a trend similar those we showed in tip #4 above.

But would it be insightful enough? A single metric trend would hide insights.

In this case it is pretty clear that Blue, Red, Green segments are doing fine. In fact the one that is absolutely most important, Green, we are doing ok.

The stink bomb in the pile is Purple. It has been dragging the overall metric down (and you know that even if the overall metric is not even shown!).

And you know how much gap you need to overcome for each segment, and were to prioritize your work (PURPLE!!).

This is just one tiny, I call it "functional", way of presenting data.

The presentation is ok, could be made more pretty.

What's precious is the process that went into creating the element – talking to leaders, meeting with Finance and Marketing, identifying the key metrics, finalizing customer segments, and establishing goals.

We often don't do all the above work (the things that are mandatory for data driven organizations) and even if we do it we don't show it because we show lame single line graphs.

Don't do that.

Kick it up a notch. You are working very hard at your job, make sure your work shows up and helps identify better insights.

Those were the five simple things you can do every day to make the most of your daily data analysis. They are not very hard to do, and they'll pay outsized dividends.

I am not someone who leaves the good enough alone. No sirree bob!

With love and affection here are 4 more bonus tips on improving your analysis and truly kicking things up a few notches:

#6: Leverage segmentation, daily.

All said and done the number one way to move from being a Reporting Squirrel to an Analysis Ninja is to leverage segmentation. Every tool has on the fly current and historical segmentation feature set. Use it.

I'll honestly not respect anyone is not applying at least some primitive segmentation to their data.

Learn how to:

~ Web Analytics Segmentation: Do Or Die, There Is No Try!
~ Google Analytics Releases Advanced Segmentation: Now Be A Ninja!

#7: Move beyond the top ten rows of data, seriously.

The "head" of your data will sustain finding insights for a month or two. You might even action something. The real gold lies in your ability to analyze tens of thousands of rows of data at one time. It is harder to do, and hence the rewards are bigger and your competitors will eat your dust more.

Learn how to:

~ Analysis Ninjas: Move Beyond The Top Ten. Find Love (/Insights)
~ Make Web Analytics Actionable: Focus On "What's Changed"

#8: Perform "pan-session" analysis, and win big.

One of the absolute criminal behaviors in web analytics (and indeed online marketing) is that we are so obsessed about Visits, and visits based analysis.

Few people sleep with you on the first date. So why is that your mental model?

Every true Analysis Ninja focuses on measuring customer behavior of one person (or in our case Unique Visitor) over the entire span of that person's interaction one our website.

Hence my devotion to measuring Days and Visits to Purchase. Truly analyzing how people buy. Or analyzing Visitor Recency and Visitor Loyalty to understand now just the first Visit (and conversion) but rather subsequent Visits (and conversions).

I tell you this is honestly kicking your web analysis up five notches, not just one.

Learn how to:

~ Measure Latent Conversions & Visitor Behavior
~ Measure Days & Visits to Purchase

#9: Evolve to multichannel analytics, achieve analytics nirvana.

There is perhaps nothing harder and nothing more impactful than getting good at multi-channel analytics.

Measuring the offline impact of your online activities gives your business a view of success that is truly orgasmic. If you get good at measuring the impact on your website of your offline activities (television, catalogs, billboards etc) then you have truly accomplished the rarest of the rate.

Learn how to: Multichannel Analytics:

~ Tracking Online Impact Of Offline Campaigns
~ Tracking Offline Conversions. 7 Best Practices.

Feeling like an Analysis Ninja already?

Of course not, you have to go do all these things! :)

Remember that tips 1 through 5 you should be able to do quite easily, just need to remember them and remember to use them. Tips 6 through 9 take time, they take a lifetime. Remember them, practice when you have time and slowly evolve over time.

Ok?

Good luck.

As usual it's your turn now.

What are your favorite tips for data analysis? When you present data what is the "trick" that you use most often to be awesome? Have you used any of the tips above? Got any favorites? What do you think it takes to morph from a Reporting Squirrel into an Analysis Ninja?

Please share your feedback / critique / tips and all via comments.

Thanks.

5 + 4 Actionable Tips To Kick Web Data Analysis Up A Notch, Or Two is a post from: Occam's Razor by Avinash Kaushik

One thing that I never want to be.

We all have a tendency to learn up to a point, we get comfortable and keep chugging along rarely investing in our ongoing education.

I call it the slow but sure path to irrelevancy.

Let me share my prescription for avoiding irrelevancy: Try new things.

Simple right?

At any given time I have six or seven interesting tools running on this website. That's not including others I actively seek out around the web. Most of them are not even related to my current job or problems I know of. And that's on purpose.

I want to constantly be in the know of new and more clever ways of working with data, tools that are often solutions to problems we don't know we have yet or tools that are sometimes seeking problems to solve!!

Irrelevancy is not fun. Stale people are not appealing (just like, as your mom taught you, a week old bread). If there is one thing you take away from it post I hope it is the importance in investing in yourself / your education / your ongoing awesomeness.

In this blog post I want to share four analytics tools that I have been playing with for a while… tools that solve an interesting problem… tools that point to what might be in terms of our near term analytical future… and in almost all cases they don't even know!

I love doing this, I hope you'll have as much fun as I do.

First Some Context.

Remember I am the creator of the 10/90 rule of investment in web analytics.

I had created the rule many years ago, early into my job at Intuit, and quite simply it states:

When I had created the rule Google Analytics did not even exist!

The rule was borne out from my own experience having inherited a world class tool we were paying $250k a year for and produced crap. Well not crap… lots of data that no one cared about or actioned. I threw out the world class tool, purchased ClickTracks for a fraction of the cost and put money into Analysts and boom!

Ok not boom overnight… but over the course of a few months the org started to be more data driven, because analysts we hired produced analysis. That fed a virtuous cycle. More analysts. More insights. More desire to be data driven.

So as you look at the tools below remember the 10/90 rule.

In the end it does not matter who has the coolest or the biggest tool. Or for that matter how many tools.

People matter.

You matter.

Remember that, at least for the rest of this post. Ok?

Let's go look at some tools…

Measuring "Invisible Virality": Tynt.

Tynt's promise is simple. Add a piece of javascript to your web page (do a View Source on this page to see it), and it will tell you how often your content is being copied.

Copied! Say it ain't so! :)

[Please click on the above image for a higher resolution version, including all the other metrics.]

In the last month data was copied off one of my posts 5,616 times, with most of it being content and some of it images.

But that's not all.

If you look at the higher resolution version (click above) you'll see it also reports other data like Visits Generated etc.

The way it works is that when someone copies a piece of content Tynt adds a little bit of additional text and a trackable code with a hash (#) at the end of the url from where content was copied.

Like so… the text that was copied from my blog is the first two lines… the Read More and link was added automatically by Tynt…

When people click on that link Tynt can report visits generated, page views, where the links were posted (in case there is a referrer) etc.

There is additional data like how many of your copies created links that were posted and then clicked on…

Gold are places were the copied text was pasted with the additional "Read more: http://…" text+link were also posted AND someone clicked on it.

You'll note that Tynt's selling point is connected to SEO. The idea that your copied text creates links back to you which in turn creates visits back to you, and per Tynt, better SEO goodness. You know links and page rank and what not!

I *personally* do not see much value in all that data. Two reasons:

So I am less enamored by Tynt data that focuses on all that.

I love the data you saw in the very first screenshot, and I absolutely love this…

[Please click on the above image for a higher resolution version, including all the other metrics.]

The first screenshot shows how often content is being copied and the above indicates the blog post / web page where the content is being copied from.

Why is this cool?

If you are a regular reader you'll notice that at the end of every blog post (before the start of the comments section) is a Topsy widget.

It measures how often a blog post is tweeted/retweeted. Goes viral. Higher the number the better, makes sense?

I also measure the # of Comments Per Post as a measure of how "engaging" / "valuable" people found the content to be. Looking at how often it was tweeted/retweeted is one more layer of information in understanding what subject / ideas in a post / things I write are well received by people and which are not.

But.

Both the above attempts measure two minorities.

The cool thing about Tynt is that it allows me to get some sense of "engagement" / "perceived value" / "Like" with the v a s t majority of people who will neither submit a comment nor write a tweet.

People who still use email. People who like something I wrote so much (or hate it so much) that they copy the text and paste it and forward it to others. Or copy the text and post it on their blogs (without attribution of course :)).

I like that a lot.

This entire interaction that was completely invisible to me is now a bit more visible. I can measure the "invisible virality" / "spread" by this big huge non-commenting, non-tweeting audience.

In the time period above I had written 4 posts (5,616 times copies). Check this out… It turns out the post with the fewest comments, just 25, and the fewest tweets, just 100…

…was copied an astonishing 506 times, when all other posts were copied in small double digits.

See what I mean… something I would have perhaps considered to be only a small success turns out was a huge hit with the blog's audience. I just would not have known that so far.

Here's another interesting application. . . Lots of people are measuring "influence" of a blogger (/ piece of content) using data from the "minority activity" (comments, retweets etc) and selling it as the complete truth. But how can you do that without some insight from the majority?

Tynt shares one very interesting piece to the puzzle that perhaps in the future fit some place where we can use it with all other context we have.

Invisible Virality. Cool right?

Applying Smarter Ideas to Measuring "Sentiment": Analyze Words.

Raise you hand if you are in the "If I am any more excited about doing sentiment analysis then I'll pee in my pants".

So many raised hands!

Here's the problem: Most solutions stink. Not just stink… dinosaur's breath after a meal stink.

We are algorithmically trying something that as yet does not lend itself to algorithmic measurement… "emotion". It is darn near impossible to cleanly buckets feelings and nuance into clean Positive, Negative, Neutral buckets.

We, computer programs, are simply not there yet. [Though I am absolutely confident that we will get there at some point.]

For now you are most likely wasting time (and money). Sorry.

Here's the other problem…

Even if it works… I don't think it works. [What!]

Let's say you have a 100% perfect human read and 100% human categorized analysis on hundreds of thousands of rows of text. Clean into the three desired categories. Like in the image above.

Now pause for a second and think… what could you do with this?

You have aggregated data into three pieces and we all know aggregated data stinks at delivering insights!

That does not mean wanting to identify insights from lots and lots of text is not prudent. It is.

I like a much more nuanced approach.

Analyze Words applies one such nuanced approach to text analysis.

It uses the well established and long use LIWC (Linguistic Inquiry and Word Count) methodology to categorize all your delightful text (in this case your tweets).

Why the LIWC? Here's the idea behind the LIWC:

Cool right?

You go to Analyze Words and you punch in your twitter id and bam (!) your "psychological" profile, or in this case mine…

Nice eh?

No simplified over promise under deliver aggregates!

The three categories and 11 sub categories provide much much much more nuanced understanding of what your text is saying, in this case for your twitter profile.

Why is this cool?

In this case measuring "Personable": Engaged in other people's well-being and at peace with expressing your own uncertainty about the world. High Scores in personable use positive emotion words, ask questions, express their own ambivalence and reference others frequently.

Better than positive, negative, neutral right?

Or "Analytic": "If law school exams were a persona, they would rank real high in this category. Ample large words and phrases that include complex thinking styles (e.g. "if – but not …")."

Love it!

Two magnificent things about this approach (remember it's not the tool, its what you do with it :))…

Of course you can do even more.

You know how you are doing… now compare it to your "competition" and find their strengths and weaknesses…

When you do competitive analysis, like above, find contrasts with your own profile, what your brand stands for in the world and their brand stands for.

Highlight differences where you brand strength is strong. Hopefully they'll discover where they stink and for the sake of humanity fix that.

Nice eh?

Analyze Words provides a glimpse of an approach that I hope others follow.

Rather than trying to find short cuts, where none exist, and provide aggregate data, where it just gets crapified, follow a well established methodology while leveraging segmentation and nuance.

We've applied it just for Twitter in the above case but you can easily see how you could apply it to call center data, tech support websites, forums, online survey open text voc answers and so much more.

Applying Simpler Ideas to Measuring "Sentiment": StatsIt.

StatsIt started off as a differentiated web analytics tool, but has morphed into a delightful social media monitoring tool.

It's approach is to index blogs and tweets and delicious and twitter and youtube and on and on and analyze that data to find yummy actionable insights about your social media presence / activity.

Like all tools it gives you pretty charts…

Sweet, now you know how much "activity" is happening. Give it to your boss, she'll be impressed. You on the other hand realize "activity" rarely has insights.

I want to focus on just one part of StatsIt that I adore because of how simple it is in its brilliance when it comes to finding insights from lots of text.

StatsIt has indexed a ton of content from all the social web activity. When you tell it your brand terms (or just your brand name, in my case "avinash kaushik") and it churns through that social web data to provide you with something awesome…. a tag cloud!

[Click on the image for a higher resolution version, along with a peek at other metrics.]

Why is this cool?

Mikko and his team have taken 1,000 words from the English language that are connected to emotion. Good emotion, bad emotion, ugly emotion.

They look at their social web data and in that they look at the words around your brand mention and finally identify the emotional words people are using in context of… you!

The tag cloud above shows the emotional words use around mentions of me for a month's worth of time.

Without having to read all the text I can at a glance now get a really good understanding of the tone and texture of activity around my presence. More importantly it does not take all that long to figure out what emotions should be there but aren't.

A very simple, effective and elegant solution to a complicated problem.

Oh and guess that happens when you click on one of the words in the tag cloud?

You are right… it takes you directly to the text from all the data that StatsIt has collected!

By clicking on the words you are essentially segmenting your data and drilling down to the text (tweets, blog posts) where you can learn more about what the person was saying when they express, say, "great" as an emotion. :)

Effective "sentiment analysis" baby!

Why can't we be this simple in other places?

We are always seeking complexity. Here are two ideas that popped into my head from the StatsIt's approach that might apply in other places.

We collect lots of open text from our online surveys right?

Rather than finding the perfect answer to what's expressed in the text, and of course getting it wrong, why don't the vendors show us a emotional tag cloud?

Can there be a better / easier / faster way to allow us to make sense of all that text, leverage as a segmentation tool and find insights every day?

Vendors! Come on!!

Another idea.

Reviews are important. Most ecommerce sites have them.

But why is it that we only see "quantitative" analysis of the reviews? 5 stars. 3.2 moons. 61% rotten tomatoes. Etc etc.

The richness of the review is only partly in the quantitative analysis of the rating. The real sweet nectar is in the words people write in reviews.

I recently gave a talk at eBay. So let's use that as an example.

You get quick quant rating on eBay that you typically use. But perhaps the real gold is here….

This seller, me, is 100% positively rated.

Now let's say that you want to buy a Sony digital camera that is listed by both me and Emer. We both have 100% positive ratings for our 60 or so prior eBay auctions.

How can you best decide if you should buy from me or Emer? You can't possibly read 120 reviews, or even scan them quickly.

Now would your life be much much easier if eBay choose to provide an "emotional tag cloud" for both Emer and Avinash?

Very quickly you could see that while we both have same quant ratings it turns out that my emotional cloud shows a neutral to positive feelings expressed while Emer's is outrageously positive.

Now is it easier to decide who to buy from?

As our dear friend Sarah Palin would say: You betcha!

So why does eBay not provide this simple emotional tag cloud?

Or for that matter Trip Advisor or Amazon or any site that hosts reviews and ratings?

Simplicity rocks. Especially when it's actionable.

Quick, Efficient, Effective Mobile Analytics: Percent Mobile.

It is always a really good idea in web analytics to understand how data is captured (case in point the delightful blog post on Competitive Intelligence data capture).

No where is this more true than when it comes to mobile analytics.

There are many methods of collecting data depending on the platform you are on, and if Steve Jobs gets upset he can totally shut you down with a mere update of his TOS! :)

I am not going to cover all that here today. For those of you who already have my second book Web Analytics 2.0 please jump to Page 250 to learn all about data collection options, platform limitations, challenges with campaign analysis and finally reports and KPI's you should measure for mobile.

In this blog post I want to share a lightweight wonderful mobile analytics platform called Percent Mobile.

Now most web analytics tools, like Google Analytics and WebTrends and others, will capture and report data for javascript enabled smart phones (like the iPhone, Android and some Nokia phones). Honestly that is all the traffic that is of commercial value, so even if you miss the rest it is not the hugest of deals.

But all these "big boys" have simply "added on" mobile analytics to their tools. The result is that they suffer from both a lack of imagination and, this is important, truly great databases when it comes to devices and carriers and other unique mobile information.

Not Percent Mobile.

They have two incredible benefits:

Here is what the resulting data looks like…

[Please click on the above image for a higher resolution version.]

No hunting and pecking to find the data, like you would in Google Analytics or Site Catalyst or CoreMetrics. A quick at a glance view of how much traffic is mobile, key stats about the devices, the devices themselves (go iPad!!), vendors and operating systems.

If you compare this to your web analytics tool you'll notice almost immediately how much better this data is compared to what the "big boys" are reporting.

Click on the image above and you'll see a bit more clearly other really sweet metrics. % of mobile devices accessing your site via WiFi. Phones with touch screens and full keyboards etc.

[Can you imagine how cool it would be to segment your mobile traffic for full keyboard phone vs none and see which convert better. Or does access via WiFi mean more content consumption than via 3G? Etc. So cool.]

That is not all… if you scroll a bit more you can get a country map view, the networks used to access your site (AT&T still #1 for me!) and countries etc.

Of course it would be hard for me to like any tool that does not allow segmentation. :) You simply drag and drop on to the box on top..

And what would an analytics tool be without the normal search, referrer and all that data we have so come to love (and hate!).

I particularly like the "Activity Types" box at the bottom left, I don't know why web analytics tools don't categorize referrers by default.

I am also surprised at the long tail of referrers. Yes Google is big but there are 91 other referrers for this segment. More mobile SEO!

Why is this cool?

It might seem odd that I would like a tool that would give me similar data that I can get out of WebTrends or Omniture or Xiti or whatever.

The first reason is that, as mentioned above, the data is actually much better because of the databases that power Percent Mobile.

The other thing is that getting this data causes less pain than pulling my two front teeth.

Finally I so do like supporting pretty tools, especially if they have good data!

The one thing Percent Mobile lacks is some way of measuring any outcomes. I can certainly dig to my "conversion pages" but it would be great if they just let me just input them into the tool and then they could measure outcomes for me (even if it is like the Goals process in GA).

But if you want a light weight easy to use free mobile analytics tool just throw Percent Mobile on your site and have fun. Go to www.percentmobile.com , click Sign Up (top right) and use the Invitation Code "Avinash" (no quotes).

Mobile rocks no?

Summary Of Our Lovely "Let's Keep Learning" Cruise.

It is important to point out that I am not affiliated in any way with any of these tools / companies. I am also not recommending overtly or covertly that you buy / use them. That is totally your call.

Of course I would not personally use them or write about them if I did not thing they had value. :)

My sincere hope is that you'll internalize:

I hope you got that, more than names of interesting tools.

I cannot tell you how much fun it is to step outside the world of Omniture and Google Analytics and other traditional web analytics tools. It stretches your mind and sometimes you look at these new techniques and data and you notice you are smiling and feel so happy.

Try it, and have fun.

[In case you were curious at the moment I am playing with these incredibly cool tools: PostRank, Next Stage Sentiment Analysis, SEO Effect, and Colligent. Each in its own way does something magical and quite unlike anyone else.]

Ok your turn now.

What do you think of the work that Tynt, Analyze Words, StatsIt & Percent Mobile do? Have you tried any of 'em? What obvious flaws did I overlook? Are there other tools you are using in the Viral, Social, Sentiment, Mobile space that you really love? If so would you please post them in comments?

Please share your feedback / critique / ideas.

Thanks.

PS:
Couple other related posts you might find interesting:

• Multiplicity: Succeed Awesomely At Web Analytics 2.0!
• Actively Avoid Insights: 4 Useful KPI Measurement Techniques
• Web Analytics Tool Selection: Three Questions to ask Yourself
• Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test
• Web Analytics Career Advice: Play In The Real World!

Viral, Social, Sentiment, Mobile: 4 Delightful Web Analytics Solutions is a post from: Occam's Razor by Avinash Kaushik