I am lucky that I get lots of interesting emails from readers of the blog. I got a one such absolutely wonderful email today from Andrew Burnette.
I am reproducing it here with Andrew’s permission with no additional commentary. I am confident you will find it as insightful and impactful as I did.
A new Malcolm Gladwell had an article in the New Yorker called Open Secrets talks about the difference between a puzzle and a mystery.
According to the article and national-security expert Gregory Treverton, the distinction is:
"Osama bin Laden's whereabouts are a puzzle. We can't find him because we don't have enough information. The key to the puzzle will probably come from someone close to bin Laden, and until we can find that source bin Laden will remain at large.
The problem of what would happen in Iraq after the toppling of Saddam Hussein was, by contrast, a mystery. It wasn't a question that had a simple, factual answer. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much."
Related to that is this one from eMarketer. A recent survey found that managers say that half the information they gather is worthless to the company
eMarketer: Useless Information: http://snipurl.com/178zk
It seems that Web Analytics probably fits into the "mystery" category. The problem is that we tend to think of it as a puzzle….that is, we think there is a "right" answer that is just waiting to be found. So we keep getting more and more information in order to find the answer–which just bogs us down. Instead, we should focus on getting the right information, and doing what we can with it.
We'll never know everything, but that doesn't mean that we can't act on what we do know and still produce results.
The Gladwell piece is worth a read if you happen to get a hold of a copy of The New Yorker. He makes a compelling argument that more data isn't always the answer.
Thanks Andrew!
What do you all think?
[Like this post? For more posts like this please click here.]
Via 
This is a brilliant post Avinash / Andrew. It goes to the heart of why most web analysts and marketing people have such a hard time with web anatlyics. They are used to solving Puzzles and have a really hard time adopting to the world of Mysteries.
It makes me think of your post from a couple weeks back about "numbers gods". We are surrounded by them and they are not the best fit for a "mystery world".
Keep up the good work with this excellent blog.
Constructing a plausible explanation for some particular web visitor behavior has got to be one of the most challenging exercises I've faced (again and again) as an analyst. It's not always possible to explain why, even with heaps of data.
By the way, Malcolm Gladwell's _The Tipping Point_ is totally worth a read.
Hi Avinash/Andrew,
I agree, great analysis is more detective work than puzzle solving. Analysts have to make judgements and recommendations using incomplete or faulty data.
However, there are many puzzles that need to be solved that will aid in solving the mysteries.
One such puzzle is identifying the best query to use to get the right evidence from your tools.
Puzzles and mysteries are different, but you have to be able to solve puzzles in order to help explain mysteries.
Great post! This is at the heart of the analysis/interpretation problem, and it is NOT addressed by applications so far. Good news: we need humans!
June, that's why we will all need in 2007 to make use of other types of analysis methodologies, such as attitudinal analysis in the case you point out.
I think we are all going to get tired of the anonymous character of huge behavioral data, and start integrating more notions of database marketing, which will make us focus on people who told us at least a little bit about who they are or how to get back to them (email, etc.). We will then be able to extract so much value, and grow a lot the number of actual and actionable recommendations we can make, that the positive perception of the analytics work will increase a lot. Watch for the dozens ways marketers will find to convince visitors to give some info!
Btw, Gladwell's New Yorker article on mysteries and puzzles is available online at:
http://www.newyorker.com/fact/content/articles/070108fa_fact
Web Analytics=Puzzle
Web Marketing=Mystery
;)
Interestingly we posted on this very theme a while back (http://www.juiceanalytics.com/weblog/?p=288). "Great minds think alike, and fools seldom differ." — Anonymous
Web Analytics can definitely be a mystery but only if the right data is collected.
Hello Avinash Sir
my website is listed in top in some travel keywords. but i dint receive any query from my website. i am getting 100s of visit that google analysis says but no query what is the reason can be. can you guide me for this situation.
thanks
Amit
Hi. This is really interesting post. Thank You! I have just subscribed to Your rss!
Best regards
Puzzle = binary code. Something that can be reassembled = linear.
Heuristic = complexity, adaptive
Both are part of the Design Thinking Continuum
http://press.harvardbusiness.org/the-design-of-business
This is an awesome framing that can cause us to focus on the Decisions rather than the Data. And to not let the search for data be an excuse for not using judgment. Thanks for retweeting this old post Avinash.
Hey, Avinash. Nice discussion!
I think that Web Analytics can be sometimes Mistery, sometimes Puzzle… just depends on who are looking for the answer!
By the way, we analysts must get the challenge: Educate! So we can turn the Mistery (client's or Hippo's view) into a Puzzle (our view, I hope).
A last thing, Avinash: you actually should keep bringing some "old" posts to twitter! :)