Imagine walking into and out of a supermarket. If you did not purchase anything then the supermarket managers probably don't even know you were there. If you purchased something, the supermarket knows something was sold (they know a bit more if you use a membership card).
Visiting a website, you leave behind a significant amount of data, whether you buy something or not. The website knows every "aisle" you walked down, everything you touched and everything you put in your cart and then discarded. If you buy, the site manager knows where you live, where you came to the website from, which promotion you are responding to, how many times you have bought before and so on. If you simply visited and left the website, it still knows everything you did and in the exact order you did it in.
Add to this the fact that now there is a massive proliferation of tools that will instantly create reports presenting data in every conceivable slice, graph, table, pivot or dump that you can imagine the challenge.
But, no matter what tool you use, the best that all this data will help you understand is What happened. It cannot, no matter how much you torture the data, tell you Why something happened. This is the reason qualitative data is so hyper important. It is the difference between 99% of the website analysis that happens that yields very little insights and the 1% that provides a window into the mind of a customer.
Combining the What (quantitative) with the Why (qualitative) will provide a company with a long term strategic competitive advantage.
There are many types of qualitative data at your disposal including brand buzz, customer satisfaction, net promoter indices, visitor engagement, stickiness, blog-pulse, etc.
While there are many options for qualitative analysis, perhaps the most important qualitative data point is how Customers/Visitors interact with your “web presence.� Visitor interaction can lead to actionable insights faster while having a richer impact on your decision making. There is a lot of "buzz" around "buzzy" metrics such as brand value / brand impact, blog-pulse, to name a couple. IMHO these "buzzy" metrics might be a sub optimal use of time/resources if we don't first have a hard core understanding of customer satisfaction and task completion on our websites.
There are many different methodologies to collect Customer qualitative data, including:
- Lab Usability Testing (inviting participants to complete tasks, guided or unguided)
- Follow Me Homes (observing in a customer's "native" environment)
- Experimentation/Testing (the latest new and cool thing to do, a/b or multivariate)
- Surveying (the grand daddy of them all)
If you are new to this world the last one is a great way to step into this new world and unlike what you might have heard it is both easy to implement, can be a continuous methodology, highly quantitative and is most often chock full of insights that will lend themselves to be very action oriented.
In future posts I hope to dive deeper into options, best practices and examples.
Agree? Disagree? Please share your feedback via comments.
Via 
Welcome to the blogosphere Avinash, I love reading your thoughts.
You're correct, web analytics is only "one" slice of how to measure the full customer or user experience. Motivation, needs, usablity, analytics, and other measurables must be taken into account.
Aside from the excellent other methods you listed, you could add 'opinion tracking' on a website.
In my experience in conducting or managing UX research The granddaddy in my opinion is not surveys –its the simple question "what do you think?"
The casual in person interview can glean the most amount of information that other methods can't track or measure –this method is too often overlooked.
Other methods:
Eye tracking
http://jeremiahthewebprophet.blogspot.com/2006/01/eyetracking-collective-intelligence-20.html
Opinion Tracking (during site session)
http://jeremiahthewebprophet.blogspot.com/2005/09/measuring-online-user-opinion-during.html
I think you've nailed the problem – it's not the "What" it's the "Why".
What I find fascinating about reading your post is the following, I can enter the (web) store and they can track everything I do, they can even ask me to fill in forms and leave some cookies on my machine… and yet, if I clear my cache and delete my cookies when I go back to the "store" they have no idea it's ME again.
This is even worse on a mobile device where I have to re-enter all my data again. Why can't "it" know it's Me and what I like. Where is the ability for the web server to "talk" to me (my device) and know more about the things (preferences) that I like?
If permission based marketing is the future then it's clear that we must understand the "why" not the what. To do that the web and it's stores have to learn that it's Me calling and what I like.
All the best,
Peter
Jeremiah: Thanks for the feedback…
Perhaps I should have been more explicit, there are many other options to get the VOC (voice of customer) to the table including the two you mention and certainly there is a hierarchy from asking my mom what she thinks of the site to Intel chips inside your brains transmitting via rfid’s exactly what our customers think of our websites (or products etc). Most of us are drowning in data and unsure of how to hear the screaming customer. Starting with simple surveys is a step towards evolving towards some of the more advanced techniques.
More importantly all the advanced techniques in the world will not add any value if you don’t understand what your core customer segments are and what the needs and wants are of each of those segments. The web is such a complex environment that anything done on a sample of six or eight or ten or twenty is rarely representative of the “real world” interacting with our websites.
Challenges galore!
Peter: Great points, couple thoughts….
As a guy whose job it is to track people I have to admit this is really a tough one. For websites to know it is You in today’s environment one of two things need to happen, #1 either the website puts something on your system that identifies you (cookie) or #2 it “reads” something that can help it figure out it is You. With both Privacy is the challenge.
#1 There is enormous hype the big brother is watching and so anonymous cookies (or cookies with your login id for example if you use My Yahoo or My Google etc) is the only way to go. If you blow cookies away the site does not know who you are, simply because it is trying to respect your privacy.
#2 This is clearly spyware. The website can install something, or read your outlook email address or your Windows login and know every time it is you. But I don’t think that is preferred by many people.
I was talking to a big seller of ringtones in the US and one problem with Mobile tracking (not just that most mobile browsers don’t accept javascript or cookies) is that the Telco is not willing to share information. For example if only they could pass the first six digits of your phone number back these guys could know better it is you. In Japan I just read of a mobile game seller sharing a cut of profits with the Telco in exchange for tracking. Maybe that is a model that will work.
If that happens in the US every time you use your mobile browser it will know it is you and it will save all your settings (hopefully with some kind of opt in method). Hooray!
Avinash,
Great to see you blogging.
One point I'd love to see you elaborate on: A/B and multivariate testing are very hot, but they all too commonly get to the what and not the why. I'd actually wager few marketers take the time to draw deeper conclusions from this, even when making a significant investment of time (if not money) in the testing.
Can't wait to read more. I'll feature your blog on mine shortly.
Hi. My name is Joel Slavis.
I represnt a small lighting company
We will need some tool to grow our business we represent many manufacturers from China and need to move products to keep them and us in momentum.
My IT people use google and the other stuff but it is not seeming to work…
Any info or help on the three
* Webside Story
* CoreMetrics
* OmniTure
or any other recommendations might be helpful
Joel: Thanks for your post….
I recommend you first read this page on my blog: Disclaimers – Disclosures to ensure you are aware of any bias that might be inserted into what I say below.
It is very difficult to make a specific recommendation without understanding your core needs and the kinds of decisions you want to make. But let me make this generic recommendation:
These are some generic thoughts, please email me if you want to know more and we can see what options are there to share knowledge.Welcome to the world of Web Insights. Good Luck!
It becomes even more challenging to consider qualitative metrics when much of our thoughts are in the subconscious (the figure of 95% is offered by Gerald Zaltman in his book How Customers Think).
As this blog demonstrates, effective web analytics (leading to insights that help reach/better objectives) is no easy matter, especially when we are dealing with the human mind…
To follow up an give a real world example on my post, consider online banking (the area I work in). A major barrier to the uptake to online services (logging in, checking your balances, transferring money, making payments, buying products) is security fears.
It's possible to build the world's most secure website, however, this will be no good if it's not usable.
You can achieve a trade-off between actual security and usability, however again, it's no good if the user is still scared.
Web analytics on their own make it hard to track the emotions involved, as we are usually talking months to years until somone first starts to use the internet, until they are comfortable performing financial transactions online. There are of course tactics to speed this along, although without offline research it's hard to gain this insight to make your web experience better…
I really enjoy reading as much as possible about the comparison between " Qualitative " and " Quantitative" measurements, specially when it is done by Avinash. I highly recommend his book " web Analytics 2.0".
I Just have a bit more aggressive point of view about the way we should be able to capture qualitative data. I don't think we would have to wait too much longer before technology allow user to have " Instant reverse engagement" with web content. Think of it as a universal short-language for web-users. This will put the users in the driver's seat and we can capture data in the areas that are important to them versus the areas we think we should test. This will let them speak when they want and about the subject they like.
Again without distracting the class I think capturing qualitative data is the next big thing and that makes the subject of social media even more important.
Matt Zamani
I really liked this article.
We mostly emphasis on quantitative part but miss qualitative, which is again most important to consider and this combination wd certainly contribute to organizational goal ..
Thanks Avinash for your nice articles & all your books.