Perfection: Perfection Is Dead, Long Live Perfection

Complex BeautyI am very shy of promoting anything on this blog, my stuff or that of others. For a number of reasons. So I'll caveat this post by saying that this is not a promotion for the book (though you won't hurt my feelings if you buy it :) ).

But I was doing a final review of Ch 13 of the book (Web Analytics: An Hour a Day, page 341 specifically) and came across this set of text.

For some reason as I read this I felt a overwhelming desire to post it on the blog today, even though I wrote this in December. Must be something in the air about data. :)

Perfection: Perfection Is Dead, Long Live Perfection.

The ever resourceful Wikipedia defines perfection as "a state of completeness and flawlessness."

As analysts, and even as decision makers, we are steeped in metrics and numbers and math and things adding up. We seek confidence in data to make decisions that can make or break our businesses (or our personal lives). More than others, we seek perfection because of our backgrounds in numbers and Excel and, most important, logic. If A plus B divided by C equals five million dollars, then we will take action Q, but only if we have utter confidence in A, B, and C.

To achieve a level of perfection, we spend more money on better tools; we slice, dice, hack, and smack the data until we feel that we understand everything about it; we spend time waiting for more data or different data; we wait for someone else to make the decision or we make no decisions at all; we lose money, time, resources, and value. It seems to make sense that it is risky to make decisions based on imperfections and that it could be expensive to make decisions when things (numbers, in our case) don't seem to all add up and perfect sense.

The problem in the real world is that nothing is perfect. It sounds obvious, but it is not quite as obvious. The challenge for web analytics specifically is that we rely on a set of deeply imperfect systems to collect data, process it, and analyze it. These imperfect methodologies include cookies, JavaScript tags, data hopping around the world passing anonymizers and firewalls, pages loading, and web structures staying the same, and so forth.

The result is that often our core human instinct to seek perfection (perfect understanding, predictability in data, stability in numbers) actively hinders our ability to find insights from our data, insights that ultimately might make or break our businesses. This is much more of a challenge for analysts because we are used to things matching up and making sense. In all of our prior experiences (in finance, ERP systems, data warehouses, business intelligence , phone sales, etc), we are used to our ability to count off numbers and apply quality controls and cleansing mechanisms that would make the data perfect (or very close to that).

The Web, on the other hand, does not make sense, in more ways than you can imagine.

Perfection on the Web is dead (well, it was never there in the first place). You will have to steel yourself for that realization and adapt your mindset to make decisions and take actions in an imperfect world. It absolutely requires some level of comfort with "faith-based analysis" to ensure that some of the sub optimal outcomes (delays, cost overruns, lack of actions, time wasted) won't happen.

Even if the pursuit of perfection is futile on the web, it is possible to make massively impactful decisions that will change your business and improve the web experience of your customers.

In this section of the book I'll provide some examples that illustrate the challenges of perfection.

I'll stop there but the story continues.

In context of everything else you'll read about data today, I hope that this tiny post was helpful.

Please share your perspective and feedback via comments.

[Like this post? For more posts like this please click here.]

Comments

  1. 1

    I'm just even more eager to get the book! :)

    Amazon says availability on April 30th… I hope I'll get it soon enough to read it on my way to eMetrics and have you sign it up!

    It's amazing how expectations are relative. People forget "classic" media, be it TV, radio or print, also has it's measurement problems. With time, people learned to "trust" TV audience or newspaper readership measurement rounded up to hundred of thousands. And suddenly, because it's the web and maybe because we look smarter than others (!), we are supposed to say the magic number is precisely X.

    Stéphane

    immeria.net

  2. 2

    Back in 2000, I started a new job at Avenue A. In our analytics team, we had three individuals who were working hard to solve a problem called "cookie death".

    These were three very polished, bright individuals. And they were absolutely frozen by not having perfect data. Consequently, they delayed products and services that would have been helpful to the businesses we supported.

    Our team took the opposite approach. We said, "what do we have that we can use?". Turns out, we had a lot. We built products and services quickly, and sold a half-million dollars of consulting work while the other team debated how to deal with "cookie death".

    All of Avinash's readers can take comfort in the fact that you still have the keys to the future of analytics. Your data, even if only 60% or 70% accurate, is far better than the data an analyst has while trying to measure the impact of a newspaper advertisement.

    Now get busy integrating your data with other corporate data (retail purchases, telephone purchases). Get busy analyzing visitor behavior over time, instead of within just one visit. You're going to be respected and gainfully employed for a very long time.

  3. 3
    ramakrishnan (Ramki) says

    We are expecting more practical web analytics problems and solutions to be discussed in this book. With the help of technology and methodology, I feel perfection is possible in web analytics.

  4. 4

    Ha! :-)

    Cheers!
    – Steve

    PS No idea what you could be referring to. ;-)
    None. What. So. Ever.

    PPS Note to self. Stop eating lunch before reading any of Avinash's blog postings. Choking at work while trying to LOL is bad form. And not healthy.

    (Added by request from a private email ;-) )

  5. 5

    Ramki: While the book is seeped in practical knowledge and very much aims to teach you "how to fish", it won't advocate striving to achieving perfection in data capture in web analytics. It will most certainly help in your, and everyone else's, efforts to understand this complex ecosystem and to extract productive and meaningful value from it.

    Perfection, or close to perfection, is possible in your approach to web analytics. It is possible in the mindset that is applied. It is possible in creating a perfect process in institutionalizing a web analytics program. It is not possible, atleast for the near future, to achieve perfection in data capture on the web.

    Debate about this is healthy, I do encourage everyone to dig deeper and understand each methodology (and the vested interests of all parties involved). Understanding is good.

    Thanks for your kind comment, I hope you'll still choose to buy the book.

    -Avinash.

  6. 6

    I moved beyond worrying about data perfection some time ago – my main challenge is educating up and down within an organization so that non-analyst's expectations are set appropriately (especially those above, who think all data should be perfect!)

  7. 7

    Great post as always. With all the recent hoopla about cookie deletion – see Comscore's announcement – your post is dead right. While you cannot achieve data perfection, you can achieve, or strive to achieve process perfection.

    Someone at the recent Search Engine Strategies conference said "The Trend is Your Friend" — I think that sums it up very nicely.

    I'm looking forward to your book!

    Chris

  8. 8

    One Comment: Can your book come out today? :) I don't want to wait for it any longer.

    I'm a perfectionist. If something is 99.9%, I want that last 0.1%. The hardest part for me with Web Analytics in general is to see data in more of a trend, and not as a deadly accurate count. Sometimes, it's as equally as difficult to explain to our customers this as well. We all want it to be perfect, but that just ain't gonna happen.

  9. 9

    Great post, Avinash!

    I make leaps of faith everyday as I never feel I understand the data as well as I should. I am always driven to reach that perfect understanding but as the overwhelming dynamics of the web and reporting tools aggregate and deadlines approach I "settle" for close enough. Those lingering doubts about accuracy of reporting are omnipresent and I am glad to see that I am not alone!

    Thanks for the thought!

    Pato

  10. 10
    Rick Rountree says

    I also struggle with the inherent uncertainty built into data coming out of analytics packages.

    I call it Conservation of Data, which is the idea that the user data coming out should be equal to the user data in. Web analytics violates the Conservation of Data, and for us perfectionists, or people who have worked with "perfect" data, it's hard to adjust.

    I keep telling myself that even though the data isn't perfect, that given long enough time periods or quantities of data, real trends emerge and can be acted upon. Kind of like looking for the signatures of elusive particles at Fermi or CERN (and only slightly less complicated ;).

  11. 11

    I think that should be "we are steeped in metrics and numbers," and not "seeped."

    Can't wait for the book!

  12. 12
    Hugh gage says

    Phew!!!

  13. 13
    Jeff Leong says

    "The problem in the real world is that nothing is perfect."

    I pre-ordered my copy.

    Looking forward to the release, Avinash!!

    Congratulations!

    Jeff

  14. 14

    Hi Avinash,

    Funny, both of the presentations we had for our first-ever Belgian Web Analytics Wednesday stressed the notion of perfection and precision.

    From Toyota's Michael Notté: don't automoate processes, try to at least capture 90% of relevant data

    From my own presentation: don't go on a crusade for precision, instead go on a crusade for quality. As a follow-up to this: don't try to measure online sales with a WA tool, your ecommerce solution does that just fine.

    So the emphasis there was on ballpark figures – if you absolutely need to present figures – and, most of all, on trend analysis.

    We have two posts on our blog that summarize this 4/18 Web Analytics Wednesday:
    a summary of the event
    a recap of the presentations (slideshare available)

    Cheers and thanks as always for your insights.

    Julien

  15. 15
    Ned Kumar says

    Avinash,
    Having been in the analytical field for more than a decade, I would agree with you whole-heartedly. Wheather it is web-analytics or traditional analytics, one can do analysis, modeling, forecasting, segmentation etc. to the nth degree — but the one thing most folks tend to forget is that like your new car, the value of your [analytical] result depreciates exponentially the more you delay getting it out (even with imperfections). There is always going to be "a better state" for any data sources you are using or tools for analyzing it. Success is the ability to use judgement and foresight and decide on what 'level' of perfection is good enough for the particular problem/issue at hand. This is where experience kicks in. For some issues, a directional answer might be enough. Whereas for another, one might have to get additional information to reduce the risk of decision.

    As always, enjoyed your blog….and waiting the release of the book :-)
    Thnx
    Ned

Trackbacks

  1. Biznology says:

    Perfection is Dead…

    I've spoken ad nauseum about how we have to stop obsessing about carefully, slowly, deciding the exact right way to do our marketing. It's comforting to believe that you have it right, but it's an unattainable dream—you never have it right. I like to say you should "do it wrong quickly," and it was gratifying to see Web metrics expert Avinash Kaushik blog about this same subject as an excerpt from his upcoming book.

    Avinash is right on when he says that perfection is dead. He talks about how there are no perfect metrics, which I've talked about previously, also. I think that some metrics experts are hung up on perfect measurements and we've tried to be more precise at the expense of unifying metrics so that people understand how to make decisions from them.

    But I think perfection is dead in an even bigger way…….

  2. […] Deux articles sur le même thème chez WebProNews et chez Avinash Kaushik: en marketing, la perfection est impossible à atteindre; La solution: "Do it wrong, quickly". Autrement dit: posez vous des questions, mais agissez avant tout; soyez réactifs et adaptez vos actions (refonte, campagne marketing) rapidement en fonction des retours. […]

  3. […]
    Persona marketing, by definition, is the application of fictional psychological and/or demographic profiles to anyone that might use a site or product. In online marketing, we’re blessed with possibly the best quality data available compared to the rest of the discipline, because we can often accurately profile anyone including prospects, clients, internal stakeholders – any visitor in fact. When we narrow the scope of online marketing to paid search or “pay-per-click” we often need to strike a balance between traffic quality and quantity. Persona marketing can help one of two ways, through the ads we craft to capture an audience, and the landing page we create to convert visitors.
    […]

Add your Perspective

*