Every believer in Web Analytics 2.0 knows that awesomeness comes not from answering just the "What" question but from also answering the "Why" question.
What comes from Google Analytics, Adobe Site Catalyst, WebTrends, CoreInsight / NetMetrics and more.
Why gives context to the What, and delightfully helps you not have to overlay your biases when you try to infer visitor intent form all the What (clickstream) data.
I know that you agree Why is important.
I know that you even realize Why is ever easier to accomplish (usability studies are economical, surveys and testing platforms start at the sweet price of free!).
Yet your site stinks like a skunk.
The reasons are complicated.
You are smart, so that is not it. Maybe it is internal politics. Maybe it is the agency you have outsourced the site to, the agency whose only competence seems to be gratuitous use of flash. Maybe it is that it is not your job, you are the "quant" guy or "GA girl". Maybe even after taking one of the team and going out on three dates the IT Dude still refuses to put Website Optimizer tags on the site. Maybe the well meaning but "never met our real customers" HiPPO dictates site design.
Bottom-line: Your site stinks and you need to fix it.
Allow me to introduce you a User Centric Design that is, I think, the solution you have been waiting for: Heuristic Evaluations
I love heuristic evaluations because they are cheap, fast and you probably already have resources you need in your company. A large part of my adoration also comes from the fact that heuristic evaluations are us going back to the basics in an attempt to create un-stinky websites.
What Are Heuristic Evaluations?
A heuristic is a rule of thumb. In as much, heuristic evaluations follow a set of well established rules (best practices) in web design and how website visitors experience websites and interact with them.
When conducting a heuristic evaluation a user researcher (or an HCI expert) acts as a website customer and attempts to complete a set of predetermined tasks related to a website's existence. For example: Trying to place and order, or looking to find out the status of an order, or the solution to an error code, or decide which of many products on a site are optimal for a specific customer persona.
But here is the lovely part, and why almost anyone can perform heuristic evaluations. in addition to best practices the researcher (or you!) will raw from their own experience, knowledge and common sense.
Heuristic evaluations are best when they are used to identify what parts of the customer experience are most broken on your website. They can be very beneficial if you have not yet conducted any usability tests or when you would like to have a quick review of prototypes that the designers might be considering.
In either case, you can quickly determine the lowest hanging fruits in terms of "broken" parts of the customer experience. With this feedback there can be iterative improvements to the customer experience. You'll probably already have connected the dots and realized that this is a fantastic way to identify ideas for A/B or Multivariate experiments on the live website.
[I Heart] Group Heuristic Evaluations
There is one more thing, a way to amplify the impact and get even better results.
Get everyone involved!
Heuristic evaluations can also be done in groups!!
Invite people around you with key skills, such a designers, information architects, web analytics professionals, that girl in accounting you really like, other analysts in the company (and their quantitative understanding of site data), search experts, the intern who only communicates via Posterous, and so on and so forth.
The goal is simple: Identify flaws by attempting to mimic the customer experience (if possible under the stewardship of a User Researcher, if not then under the gaze of your haunting brown eyes) by completing the tasks on the website as a custom.
The great benefit of the group heuristic evaluation method is that you can tap into the "wisdom of crowds". This is especially powerful because the web is such an intensely personal medium, and the group members can offer different points of view that highlight issues quickly.
The process that worked optimally for me was to send an email to 10 or so folks (a diverse set!). Invite them to a noon meeting in a largish conference room and order lunch for them (best $50 I ever spent). Once everyone was settled (by 1205!) project the website on the screen and try to complete the most common customer tasks.
I (or you) have to do a good job of moderating the discussion and ensure everyone participates. There is no such thing as a bad opinion, diversity is good. Collect all pertinent feedback.
Heuristic evaluations can provide valuable feedback at a low cost ($50 in my case) in a very short amount of time (an hour in my case) and identify obvious usability problems. Hence they are best for optimizing work flows, improving user interface design and for understanding the overall stinkiness (or lack thereof) of the website.
Here is another subtle benefit of the group evaluations: improved communication and, dare I say, camaraderie between different groups in your company (big or small).
There is nothing that quite brings people together like a, pardon the expression, bitchfest. Everyone contributes, everyone commiserates, everyone loves it. Next time they are doing something they'll know what you do. Next time you need help, you'll know who to call (and they'll pick up the phone!). It is a great way to bring a sense of common purpose and a sense of ownership to improving the website experience.
Conducting A Heuristic Evaluation (The Glorious Process!).
You're excited right?
Here are six steps to conducting a successful evaluation process, either when you do it or you are doing it as a group:
1. Write down the tasks that customers are expected to complete on the website. If you are using surveys on your site (even a simple site level survey like 4Q from iPerceptions or page level survey like Kissinsights) then that is a fantastic source of this information. You should also, if possible, talk to the site owner. Here is what you might end up with on your list:
- Find information about the top-selling product on the website.
- Locate a store closest to where the customer lives.
- Place an order on the website by using PayPal. (If the website doesn't accept PayPal, how easily and quickly can a customer find that out?)
- Sign up to show up at a protest march against taxes on the richest Americans.
- Successfully contact tech support via email.
- Pick the right product for customer profile x (where x can be a small business owner or a family of four or someone who is allergic to peanuts).
2. This is sometimes hard but try to establish success benchmarks for each task. For example success rate for placing an order for the best selling product = 80%, signing up for the protest march = 99%, contact tech support = 90%, etc etc.
3. The fun part. Walk through each task as a customer would and make note of the key findings from the experience – everything from how long it takes to complete the task to how many steps it takes to the hurdles in accomplishing the tasks to how profound your embarrassment was that this was your own company's website.
4. If you were using a best practices checklist (more on this below) then make a note of the specific rule violations.
5. The hard part. Create a report of the findings. You can use PowerPoint with a screenshot of the webpage and clear call-outs for issues found. Or you can use Camtesia / a screen recording software to capture the session (and the group discussion). This can be distilled to a "the best of the bitchfest" collection for your superiors.
6. The hardest part. Categorize the recommendations into Urgent, Important and Nice to Have. We all get swept into emotions fervor. It is also possible that when you present your findings to your Sr. Management they might be a bit HiPPOish (not that there's anything wrong with that).
You want to go in with the Urgent, Important and Nice to Have based on impact on the customer experience and the company bottom-line. This helps drive a "what we should do" discussion rather than "I think we should do that" discussion.
The process, as always in web analytics, is important and I hope the above six steps help you create a process in your company that is repeatable and yields impactful results.
I want to stress again that this is a great way for you to get into the customer's shoes, for your to build camaraderie, involve the cross-functional team of people, and finally find the lowest hanging fruit for sure and perhaps even some big ones.
The Cheapest Heuristic Evaluation Exercise.
I have hinted about the cheapest possible heuristic evaluation exercise a couple of times in this post.
It is: You sitting down with your common sense and a list of "best practices" and checking how well, or badly, your website does against that list. Then do steps 5 & 6 above.
This is fast and impactful. Even in the worst case you identify the most broken things / annoyances.
I have used lots of website best practices usability checklists over time and have settled on using Dr. Peter Meyers's 25-point checklist. It is simple, effective and quite expansive. You can download it here. [If you' have Web Analytics 2.0 then you'll also find an "extended edition" on the CD that is attached to the back cover of the book.]
The usability checklist has four sections. Here's a brief summary:
1. Site Load-time Is Reasonable
2. Adequate Text-to-Background Contrast
3. Font Size/Spacing Is Easy to Read
4. Flash & Add-ons Are Used Sparingly
5. Images Have Appropriate ALT Tags
6. Site Has Custom Not-found/404 Page
7. Company Logo Is Prominently Placed
8. Tagline Makes Company's Purpose Clear
9. Home-page Is Digestible In 5 Seconds
10. Clear Path to Company Information
11. Clear Path to Contact Information
12. Main Navigation Is Easily Identifiable
13. Navigation Labels Are Clear & Concise
14. Number of Buttons/Links Is Reasonable
15. Company Logo Is Linked to Home-page
16. Links Are Consistent & Easy to Identify
17. Site Search Is Easy to Access
18. Major Headings Are Clear & Descriptive
19. Critical Content Is Above The Fold
20. Styles & Colors Are Consistent
21. Emphasis (bold, etc.) Is Used Sparingly
22. Ads & Pop-ups Are Unobtrusive
23. Main Copy Is Concise & Explanatory
24. URLs Are Meaningful & User-friendly
25. HTML Page Titles Are Explanatory
Seems simple right?
I bet your website currently breaks 10 of the rules above. It is hard to believe. Set up a quite hour aside. Go through the checklist. But first go download the detailed checklist at Dr. Pete's website.
When you are done remember to do steps 5 and 6 of the recommended heuristic evaluation process outlined above.
Benefits of Heuristic Evaluations.
In case you somehow made it here and were not convinced of the value doing heuristic evaluations here is a quick summary of the benefits:
- Heuristic evaluations are extremely fast to perform, with a very short time to insights.
- They leverage existing resources in your company (what could be awesomer?).
- You'll identify the most egregious customer issues on your website (often all the low and medium hanging fruit, quickly).
- They can be used very effectively early in the website development process to find potential customer hurdles / deal breakers.
- If you have an existing UCD program or hire an external company/agency. heuristic evaluations can reduce the cost of full usability tests by helping fix the obvious problems. The $900 an hour charged by the Agency can then be focused on hidden / really tough challenges.
Thing to Watch For.
It should be clear that I am a fan of the heuristic evaluation process. And it is every fan's duty to also highlight things to watch out for. Here they are:
- Both single person led or group evaluations contain company employees, and sometimes usability experts, but none of them are the actual customers. Despite using best practices and our wisdom we might miss some subtle problems, even as we identify the obvious ones. Be very aware of this.
- [It follows from the above point that.] The better you are at step #1 outlined in the heuristic evaluation process, the better your outcomes will be.
- When there is disagreement in recommendations from the heuristic evaluations there can be great value by doing live website tests or usability studies (whatever is faster, usually experiments).
- Heuristic evaluations are best for optimizing work flows, larger more obvious parts of website design and the overall usability of the website.
In summary: Not quite God's gift to humanity, but rather the best thing you could do to identify the low and medium fixes to your site that will significantly improve the experience of your customers.
Don't spend your day immersed in Google Analytics and just the "What" analysis. Understanding "Why" is the key, use it to unlock actionable insights.
Ok your turn now.
Have you done heuristic evaluations for your website? Who leads them in your company? Do you have a list of usability best practices that you use on your website? What other methods of of listening / collecting voice of customer / answering "Why" do you use in your company?
Please share your feedback / critique / questions / answers via comments.
Couple other related posts you might find interesting: