Measuring Online Engagement: What Role Does Web Analytics Play?

engagementEngagement is a buzz word. It is a quest. It is altar at which many worship.

Often though, atleast online, our hopes are dashed, efforts expended rarely have adequate ROI, the hype is followed with a bucket of cold water.

It is not that measuring if "Visitors" / "Customers" itself is a ignoble goal. It is more that our execution efforts in measuring engagement are fatally flawed. So much so that recently I was compelled to pen a warning post: “Engagement” Is Not A Metric, It’s An Excuse.

That post outlined the core issues faced in the quest of measuring Online Engagement. It also outlined a four step process you should follow when it comes to trying to measure engagement.

One of my central thoughts was that it is nearly difficult, if not near impossible, to measure Engagement, as many others have passionately recommended, based only on your quantitative data (in our case Web Analytics clickstream data).

So it was with great delight that I read Theo's email that helped me understand exactly why I felt that way! :)

When Theo proposed posting it here on Occam's Razor I quickly agreed.

Theo Papadakis is a Marketing Executive at cScape in London. Our paths first crossed when he invited me to contribute a thought piece to accompany the 2nd Annual Customer Engagement Survey Report (and to his credit he did so after having read my post above!). You can download the report, there are nice graphs and some interesting data and page twenty is quite interesting. :)

Here's Theo. . . . . .

__________________________________________________

Measuring Online Engagement: What Role Does Web Analytics Play?

Before we look at what aspects of a customer's online engagement Web Analytics can capture we need to clarify the meaning of the concept of customer engagement.

Definition of Customer Engagement (Engagement Index)

'Engagement' is a word with many meanings (vow, betrothal, involvement etc). For marketing purposes they can be boiled down to a single concept: one-way relation. If x is engaged with y, x is related to y.

The concept of customer engagement only deals with a particular kind of one-way relationship:

  • Subject of engagement: The subject of engagement should not be limited to customers.


    Although 'visitor engagement' is better in that it takes into account non-customer visitors to your website/store, its focus on measuring people's engagement with your brand on your own premises is too restrictive.

    It is important to measure the engagement of customers, prospective customers and detractors with our brand, in every space they engage with it in.

  • Object of engagement:

    The subject's relationship with a brand / company / product / consumption topic.

Now that we have defined what kind of relationships customer engagement deals with let's look at the criteria with which we can refine and classify the ways in which customers engage:

  • Kind: Customers can be positively or negatively engaged with a company/product.

    A more in-depth examination of kind would reveal its content, usually a mixture of emotional states and rational beliefs, such as, in the case of positive engagement, sympathy, trust, pride, etc

  • Degree: The degree of positive or negative engagement lies on a continuum that ranges from low involvement, namely, the psychological state of apathy, to high.

    An engaged person is someone with an above average involvement with his or her object of relatedness.

With the context setting out of the way. . . . .

What aspect of customer engagement can web analytics capture?

Having defined customer engagement we are better able to delimit what web analytics can and cannot tell us about the engagement of our website's visitors.

Let's look at some of the widely used web analytics metrics and understand what aspect of engagement they capture.

Unique Visits: Shows how many people decided to engage with you for the first time by visiting your website.

Frequency of Visit: Frequency must be contextualised within a specific time frame.

A customer who has engaged 10 times with the company in the past 10 years has a lower degree of engagement for example in relation to a customer who has also engaged 10 times in the company in the last 2 months.

Contextualised 'frequency' can therefore help us to identify the relative degree of our customers' engagement.

Recency of Visit: This metric speaks of the recency of our customers' last engagement.

recency of visit google analytics 1

Jim Novo has proven that it correlates well with degree of engagement. A customer whose last engagement with a brand is more recent than that of another is also likely to be more engaged.

Like frequency it therefore frames our customers' degree of engagement only relatively.

Depth of Visit: This tells us how many pages long our visitors' journeys through the site were.

depth of visit google analytics

Although a deep journey signifies a high degree of engagement this metric again does not distinguish between the kind of engagement.

Do your visitors passionately disagree with what you are writing about? Are they simply unable to find what they are looking for?

In both of these cases a high degree of engagement may be of a negative kind.

Time Spent on Site: Same story as with depth. Time spent correlates with degree of engagement but as it does not discriminate between kind it may simply be negatively spent desperately trying to find the content your visitor is after.

Similarly, most online metrics are only able to capture degree not kind of engagement:

  • Subscribing (feed, email, newsletter)
  • Registering
  • Feedback (comments, complaints, inquiries etc)
  • Rating\tagging\filtering\bookmarking its content
  • User submissions (UGC)
  • Printing or downloading a piece of content
  • Brand index

Degree of Engagement

What comes out of the above discussion is that

…it is impossible to derive the kind (positive/negative) of your visitor's engagement using web analytics alone, and, therefore, that…

…when we are talking about customer engagement in the context of web analytics, we are in fact talking about degree of engagement.

This is not to say that we cannot make inferences and state hypotheses about the kind/content of engagement, based on what we can measure (degree of engagement), nor that these hypotheses are unlikely to be correct.

It is only to say that using web analytics it is impossible to make or support such inferences.

Such inferences, about the kind of engagement, must necessarily be informed by considerations that lie entirely outside the field of web analytics.

Before we begin making inferences on the basis of degree of engagement however let's discuss this metric a bit more.

45degrees 1Following a number of leading Web Analysts I also believe that a customer's degree of engagement is better calculated as a synthetic metric composed of several basic metrics, rather than as a one-metric solution e.g. measuring customer engagement by means of 'duration of visit' only. This requires an argument unto itself that will not be pursued here.

The score each of these component metrics takes however only makes sense if contextualised. Example: a frequent and recent visitor is 'more engaged than' someone who is not, but is he engaged? If yes how engaged is he? There is little we can do with relative statements such as this.

In order to make such statements meaningful and operational we need to contextualize the component metrics that constitute a customer's degree of engagement on a high/low continuum, beginning with apathy and proceeding with progressively higher degrees of engagement.

This means that both the lowest (apathy) and the highest degree of engagement need to be defined. The easiest way to do this is to define the average degree of engagement (the average score for several metrics of your choice across your site or based on a competitor-specific or industry-wide benchmark), considering everything that falls short of it as (increasing degrees of) apathy and everything beyond it as (increasing degrees of) engagement. In this way a customer's degree of engagement assumes a non-relative meaning (it remains of course relative to your website's, competitor's or industry's historical performance).

x greater than y

By inserting relative statements such as 'x is more engaged than y if and only if x does z and y does >z' into a continuum that is based on website \ competitor \ industry benchmarks, it is possible to provide a reference point which although relative in itself (historical performance) is sufficiently stable and pertinent to business performance, to provide with useful insights into visitor behaviour and business\campaign success. (Substitute z with any or an aggregate of the visit metrics score(s)).

Conclusions

  • No web metric, or combination of metrics, can discriminate between kind of engagement i.e. positive engagement. This requires primary research.

  • All web metrics can do is discriminate between relative degrees of engagement.

  • Basic metrics can only discriminate between low degrees of engagement.

  • A customer with a high score in his visit metrics may nevertheless feel apathetic towards the brand.

__________________________________________________

That was interesting, was it not? You can now see why I found Theo's article to be so enlightening.

I found the nuance of the kind of engagement and the degree of engagement to be particularly insightful.

The next time I take quantitative data (even if I mash it into a formula of five different metrics and call it website engagement index!) to my C-level executives I will first state what my engagement index measures is Degree of Engagement.

My hope is this will result in a clear understanding of the limits of what the data is saying.

That then I hope will lead to questions about measuring the Kind on Engagement (and to exploring qualitative measures).

No false promises made from the data. Progress made by understanding its limits and exposing them. That's awesomeness. IMHO.

My heartfelt thanks to Theo for sharing this article with me, and now with all of us.

It's your turn now.

Agree? Disagree? What's your experience? Please share your perspectives, critique, bouquets and brickbats via comments.

PS:
Couple other related posts:

Comments

  1. 1
    Steve says:

    Thought Provoking and Insightful. Thanks Theo & Avinash!
    I look forward with fascination to the discussion(s). :-)

    Cheers!
    – Steve

  2. 2
    Nick P says:

    Appreciate we could argue about the definition of engagement for hours, however I'm not sure I agree with the statement:

    The concept of customer engagement only deals with a particular kind of one-way relationship

    To my mind if someone has a high degree of engagement with your site/brand, then using the measures supplied – recency, frequency, depth, etc – it implies that your content and copy is hitting the mark. Its correctly targeted at your audience and they find your site interesting and come back for more.

    So if you're putting content out there and your visitors are coming back and showing interest in it, does it not imply that any engagement score is more a measure of the depth of the relationship you have with your visitors? If so, this is two way – the information/services you are providing and the interest (engagement) that your visitors are showing in return?

  3. 3
    Jaume Clotet says:

    Hi Avinahs,

    I find this post very interesting, let me participate on the concept but without drilling down to the formulas.

    I see engagement is a concept used in marketing to predict changes on the relation with our visitors, then a necessary concept. Well, as a marketing concept sometimes is more near to art than science but in fact we need a kind of 'metric' to measure the relation with visits, visitors and customers in terms of 'engagement'. I will be more close to the art if you allow me ;-)

    In my experience 'engagement' must be related to some segmentation. I don’t see the qualification between visits and others (visitors, frequent visitor, customers, loyal customer,..) as tipes of engagements, but probably ‘engaged’ visits will come back and ‘engaged’ visitors are more willing to become customers. In my opinion every group have their particular type of engagement.

    Then engagement for visits should be to stay or not when arrives to the web, and the degree should be measured by time (or page views if its possible). In terms of visitors the engagement should be to repeat or not, and the degree should be measured by repeating times (frequency or recency). In the case of customers we have other ‘metrics’ that help us to measure relation in terms of sales but not in terms of ‘engagement’, we can have a very loyal customer (repeating buying) but with low ‘engagement’ in terms of relation with our web, other hand we should have a non repeating customer with a high ‘engagement’ as a visitor. In this way we should create different groups to predict and to focus communication with them.

    Well in conclusion my argument is try to define ‘engagement’ for the different stadiums of relation, in each stadium engagement means different thinks and probably needs specific metrics to measure.

    All the best!

    Jaume Clotet

  4. 4

    My experience has been that when people start trotting out the "engagement" spiel, you should:

    * Start listening more critically
    * Understand and question the assumptions they are making
    * Most importantly–watch your wallet

    I've personally heard it as both an excuse for missing metrics and as a deflection tool.

    For the former, it's used in "yeah, I missed the profit number, but my users are more engaged as you can see by the following, so please give me a pass, because it will get better later…" For the latter, it's "I hit my top of funnel numbers (such as registrations or UVs), but [insert somebody else here] wasn't providing the content or systems to enable us to achieve the desired metrics."

    I agree with you that there's not as hard a definition of "engagement" as with other metrics such as UV or PV and, as long time readers of your blog will recognize, even those can be debated. The fact that engagement can be squishy causes programming people to do odd things. Like photo galleries (see your big portals). Or refreshing pages more frequently (see Drudge). And there's tons of others.

    At this point, I'm leery of spending a lot of time on engagement metrics, when I can focus on trying to measure LTV. Not to say it's a waste of time to measure engagement, but it's so variable even within your company–which has many products and services, typically–that using the LCD of profit short cuts the debates.

    As always, thought-provoking!

  5. 5
    Sean Burton says:

    Having worked with cScape for some time, it's great to see this kind of methodology being mentioned here.

    Being able to define engagement metrics for a site is absolutely essential, but in the shorter term for individual visit (Visit Scoring), and in the longer term (Visitor Engagement / Scoring / Propensity).

    As with previous posts, there is a depth to web analytic data which now includes so much more than simple traffic data (how I wish that 'simple' were the case!! ;)). Using qualitative data to enrich the traffic data, along with performance, commercial, and customer data can lead to a significantly deeper level of analysis.

    This kind of integrated approach can provide extremely interesting results when using off-line data feeds as well. Here you can start to find customers that use the site regularly, but only purchase infrequently or via a different channel, such as off-line via the phone. Using these data you can then start to understand how different customer segments want to use the site, and how your customers want to interact with you as a brand rather than simply the web site.

    All of the above depends, of course, on accurate visitor identification over time – usually in the form of a unique registration/user identifier. Whilst this cannot be used directly to identify a person, it can be used to link data sources together, providing a much more tailored approach.

    The alternative is to use some form of persistent cookie. However, research has shown that cookies become increasing less reliable the longer the time period that is being used (See Jupiter Research: http://www.jupitermedia.com/corporate/releases/05.03.14-newjupresearch.html).

    We are currently in the process of investigating Cookie Deletion, along with Cookie Blocking and Ad Blocking, so that we can provide feedback on how these factors may be affecting web metrics such as unique visitors and visitor RFML (Recency, Frequency, Monetary, and Latency) over different time frames.

    Sorry rant over :)

    Look forward to tracking the topic over the next few days and week!

    Cheers,
    Sean.

  6. 6

    Jaume: I fully agree with you that we need to use engagement to segment audiences.

    The problem i want to point to is that it is sometimes hard to differentiate, and hence segment, between a very engaged customer with a high emotional connection to your brand who nevertheless does not contribute content (comment, rate, provide feedback etc) and another whose behaviour is identical from a web analytics point of view with the former, but who is engaged very little or even negatively.

  7. 7
    S.Hamel says:

    The concept goes back to Jim Novo's fabulous concept of Momentum & Friction and the concept of RFMA I introduced a couple of years ago. Using the traditional Recency, Frequency, Monetary value and the concept of Attention as a segmentation method for measuring visitor/customer value/engagement.

    The thing is, in your example with Google Analytics, we get Recency, Depth and Time Spent, but we can't establish any correlation between them, thus making this evaluation impossible with Google Analytics. Furthermore, Jim Novo's concept uses Frequency (how many time), Recency (how long ago) and Latency (how long between two significant events). Sadly, Latency can't be measured with Google Analytics (as with many other WA tools on the market).

    We've been trying to re-invent the Engagement metric and it's been there all the time with Jim's Friction model and RFMA. The problem is a) we don't agree on a simple formula b) it's still hard to get the base metrics to compute it with the actual tools on the market.

    Best regards,
    Stephane Hamel

  8. 8
    Jaume Clotet says:

    Hi Theo,

    Well, in my opinion engagement is something more related to ‘willing’ and ‘action’ than to ‘emotion’ and ‘intention’. Is like when we are in love (yes, call me romantic) and you think a lot with the other person about possible plans, possible future and possible everything but you never moves, always another better thing happens instead what you wished… to be honest then, we are ‘in love’ but not ‘engaged’. Obviously, if we are trying to measure how ‘in love’ we are that is not possible, but ‘engagement’ should be measured related to everything we have done together.

    Then, going back to the problem “differentiate between a very engaged customer with a high emotional connection to your brand who nevertheless does not contribute content (comment, rate, provide feedback etc) and another whose behaviour is identical from a web analytics point of view with the former, but who is engaged very little or even negatively.” In my opinion those who nevertheless does not contribute are not engaged :-)

  9. 9
    Jim Novo says:

    Most every marketing metric can be used at both a strategic and tactical level, the difference typically being the time frame.

    In the case of engagement, we keep constructing all kinds of models that try to ram the strategic and tactical together, which is suboptimal, because they are different ideas.

    If you are talking about measuring engagement with a web site during a visit, that's tactical. If you are talking about visitor actions over time, that's strategic.

    Clearly, I can visit a web site for the first time and be very engaged, going through it in detail and trying all the bells and whistles. But then I can decide the site sucks and never go back.

    If you are simply looking at "in visit" engagement, a series of one-off interactions unrelated to each other over time, that's a tactical approach, and it's fine, as far as it goes, though it doesn't provide the predictive element available in the more strategic approach.

    Whether or not you care if visitors come back to your site is a business model thing, but for the majority of sites, I would think it's of critical strategic importance, so looking at engagement over time, rather than "in visit", is the more important idea.

    In other words, many web analysts focus on answering this tactical question:

    What did they do?

    BI / Marketing folks are more interested in answering this strategic question:

    Will they continue the relationship?

    This difference, I believe, is at the heart of a lot of the conflict in the engagement discussion.

    In the context of Theo's piece, the tactical "in visit" approach is about degree of engagement, the strategic "over time" approach is about the kind of engagement, and more importantly, predicting the kind of engagement in advance so that optimization can take place *before* the evidence of dis-engagement has been collected.

    In other words, the strategic approach is pro-active, as opposed to reacting after the fact.

  10. 10

    I think we've all overlooked what the Web Analytics Association calls a dimension which should be taken into account.

    The WAA is after all trying to put to bed the "semantic" issues surrounding the industry.

    I would welcome comments to my own method of dealing with the issue.

    http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model

  11. 11
    Eric T. Peterson says:

    Hi Avinash,

    I have to say I'm pretty surprised to read this post — you seemed to be pretty adamantly opposed to the idea of an operational measure of visitor engagement based on quantitative and qualitative data. But your statement …

    "Following a number of leading Web Analysts I also believe that a customer’s degree of engagement is better calculated as a synthetic metric composed of several basic metrics, rather than as a one-metric solution e.g. measuring customer engagement by means of ‘duration of visit’ only."

    … appears to be a retraction of your previous assertion that the creation of an 'Engagement Index' is an expensive waste of time. Your comment to Steve Jackson:

    "We could hire a Consultant who will spend three months and create a complex “Engagement Index” for you that has weighted average/whatever and put all the above metrics in that Index.

    They walk out of the door with US $200,000 and you are left holding a “Engagement Index” that no one understands broadly and when the Index goes up or down no one has any idea why and what needs to be done.

    The Consultant is happy, you are left holding the bag with a hole (where the $200 grand was)"

    (I should comment, if you're getting $200K to do this kind of work, I am certainly impressed and clearly in the wrong business. LOL!)

    Anyway, I'm glad to see you coming around! I personally (and fairly strongly) believe that an operational measure of visitor engagement is a critical few KPI for Web Analytics 2.0, Web Analytics 3.0, and beyond. And the good news is that my work is being ground-truthed with great results around the world!

    Your readers who would like to study a synthetic metric composed of several basic metrics are welcome to visit my site and read "How to measure visitor engagement, redux" (blog.webanalyticsdemystified.com/weblog/2007/10/how-to-measure-visitor-engagement-redux.html)

    Or, as Joseph pointed out (altho his link is broken for some reason), you can join us in debate on the subject (and Avinash, we would LOVE to have your perspective!) at:

    thefutureof.webanalyticsdemystified.com/2008/01/28/starting-the-discussion-attention-engagement-authority-influence/

    Finally, if you would like to discuss measures of engagement face-to-face, I will be in San Jose on Thursday and would be happy to meet if you're in town. You know how to contact me.

    Again, it was refreshing to read your re-evaluation of engagement and I'm glad to see you've changed your mind! Nice shout to Novo as well.

    Sincerely,

    Eric T. Peterson
    Web Analytics Demystified, Inc.

    Note : Eric: I think you meant to address this comment to Theo, the post represents his kind position and valuable perspective. Thanks as always. -Avinash.

  12. 12

    Howdy,
    I know readers of this blog will be shocked to learn that I have a different idea of "engagement".
    For what it's worth, Eric Peterson and I have been discussing attention, engagement, and trust at http://thefutureof.webanalyticsdemystified.com/2008/01/28/starting-the-discussion-attention-engagement-authority-influence/

    . I'm enjoying the discussion (can't speak for Eric)and welcome the opinion of others. (As I'm admittedly learning how web analytics think, I'm desperate for the opinions of others) – Joseph

  13. 13

    Mark: The examples of misuse and abuse of 'engagement' are part of the motivation behind this post i.e. clarifying the concept\phenomenon and in turn identifying how far web analytics can take us on the way to capture\understand and use it (predictive value, segmentation, campaign evaluation etc).

    As regards the value of this metric I would point you to Jim Novo's work which is the best in terms of ROI. Some of his suggestions are incredibly simple (his use of recency for example) but provide with spectacular results. For a more general\theoretical account of the value of engagement check a post i wrote: http://marketingroi.wordpress.com/2007/11/30/the-value-of-customer-engagement/

  14. 14

    Nick: I only meant to say that customer engagement deals with understanding\measuring\predicting\segmenting the customer's relationship with the brand\organisation.

    Perhaps, this is not what you meant, but what you said made me think as to whether, at some point in the future, and as direct employee-to-customer engagements 'a la' Cluetrain proliferate (i.e. not centraly moderated and brought under the uniform brand identity- umbrella) the kind/degree of employee-to-customer interactions will need to be measured as well.

  15. 15

    I got to admit that I don't really agree with you. I think that web analytics tools can measure engagement. They just need to measure more things besides page views and uniques.
    This is exactly what we do in my company (NuConomy).
    Here is a short example:

    User A, goes into YouTube. He goes through a few channels and user pages. He browse about 30 pages which brings him deep into the site, but he doesn't watch even a single video.

    User B goes into YouTube. He watch the top 5 videos in the home page. He rate 3 of them and make comments on the other two. He than continue to upload two videos of its own. During all this time, he doesn't browse more than 5 pages in the site.

    Now – Which of the users do you think is more engaged with the YouTube brand?

    I won't talk here too much on what we do with our product as I don't want to sound like an ad, but we do beleive that this is the future of web analytics. The ability to measure all the rich interactivity that happens on a site, and analyse it in meaningfull ways.

  16. 16
    Chuck Ullan says:

    Lots of ideas here and to some extent, everyone is right. Or no one is right.

    In my view, engagement is not an industry metric, period, or at least one that can be comparable across companies. I like to say that all engagement is local.

    I view it as a philosophy. An aggregation of relevant, desired activity.

    My experience at a large portal is that the metrics we'd consider engagement vary wildly across our own sites. Whether the metric is internal or external, it comes back to "what do I want people to do more of?" and to the extent you can measure it, great.

    In that view, the lists of options that Eric Peterson and Avinash put together are great. It's basically your shopping list of stuff your you to think about in the context of your own site. How that's ever comparable for an industry beats me.

    Sentiment (positive/negative point of view) seems like a different learning, which could borrow a few of the same construction components from engagement, but would require different pieces as well.

  17. 17

    Avinash,

    I'm confused: what's the purpose of using Engagement metric? What problem does it help to solve?

  18. 18
    Eric T. Peterson says:

    Hmm, my comment from earlier today seems to have been deleted. How's that for engagement!

    All the best,

    Eric T. Peterson

  19. 19
    Eric T. Peterson says:

    Oh wait, I see now! This isn't Avinash's position at all … that makes more sense! It was difficult to discern because Theo's post looks so much like your own posts (with the images and all … nice looking work.)

    Nice of you to publish a conflicting opinion, Avinash.

    All the best,

    E.

  20. 20

    Oops, now it's back! So strange!!!

  21. 21

    Everyone : My replies below. Apologies for the delays in replying to your emails and moderating comments. I am in Atlanta this week, two important speeches today – six hours in total, requiring sixty hours worth of energy! If anything below sounds sub-optimal, I blame exhaustion!!

    Thanks so much for your feedback and generous comments. I owe a debt of gratitude to Theo for his wonderful article, and now to you for pushing the conversation even further along. Thank you.

    Nick : Measuring the Degree of Engagement is good and can provide conversation starters with decision makers.

    But I humbly believe that it is squishy to assume that a "high degree" of engagement (via frequency or depth) implies you are "hitting the mark".

    Hitting the mark would mean that the kind of engagement is positive. While this may be true in some cases, it is a leap of faith. It could just as well be negative. Just a thought.

    Jaume : I think your suggestion of complementing clickstream data with other sources (surveys – onsite or email – and primary or secondary market research etc) will fill in the "probably engaged" hypotheses.

    Sean : If I am reading Theo right then there is no implication there that Engagement is not important, just a stress on what can you get just from having clickstream data.

    Most formulas and proposals our there still use, and aggressively push, pure clickstream data as the end all.

    You have my vote when you say this: "there is a depth to web analytic data which now includes so much more than simple traffic data (how I wish that ’simple’ were the case!! ;)). Using qualitative data to enrich the traffic data". I have stressed this quite a bit in my earlier post on engagement as well.

    As you can expect, I, along with others, anxiously await your cookie report!!

    Jim : Thanks for adding the context and detail Jim, as always it is very interesting.

    I have to admit that after having done this for some time (baby years compared to your experience though) and will so many different clients, I remain unconvinced that web analytics by itself can give any signal when it comes to Kind. In fact the further back you stretch in time the worse the problem becomes.

    This is Not the case for offline data (catalogs, call centers, crm systems etc etc), atleast it is less of a problem.

    I am convinced this will change with time but for now, and this is the reason Theo's piece was so crystallizing for me, I can take a clean understand of what we can give, degree, and if asked for kind then I can explore other avenues (qualitative measurement, tying historical actual customer data – not cookie based – from erp / crm systems etc etc).

    Joseph : Congratulations on the launch of the new blog!! I have added it to my RSS reader and look forward to reading the insights.

    Shahar : You are right, web analytics tools can measure engagement. They can measure Degree of Engagement!

    Ok that is a bit cheeky!! :)

    Great to hear of the work you are doing, mixing clickstream with qualitative and on site as well as off site behavior. Very interesting.

    Dennis : Your comment just stopped me in my tracks. Brilliant, just brilliant.

    My reply: Ask those two question to every single person (your boss, a consultant, a neighbor, a blogger, anyone) who suggests you measure engagement.

    I promise you their answers will be deeply insightful, throw in a pinch skepticism and then even if you measure "engagement" I think you'll be on the right track.

    I loved your comment!

    -Avinash.

  22. 22
    Nick P says:

    Shahar's comment reminded me of WebTrends Score which I have previously wondered about using to measure engagement.

    Basically you assign (arbitary) points to various actions on your site. This takes the measurement beyond visits, views and uniques allowing you to grade some actions on your site as more important than others. You can then segment your visitor base by their scores, those with higher scores having a greater degree of engagement (or whatever we're calling it).

    (I appreciate other analytics solutions can probably do the same, however I'm just more familiar with WebTrends – hence the example from them!)

  23. 23

    Jim: I agree with you that focusing on the ‘what did they do of visitors’ is of very little value unless it informs the ‘will they continue the relationship’. The predictive value of customer engagement metrics must be used to inform pro-active strategic choices.

    However the ‘will they continue the relationship’ is based on the ‘what did they do’.
    The predictive value of the engagement metrics is based on discerning behavioural patterns in behavioural data (which by definition is of the past).

    Frankly I am not sure I have understood your comment. Can you please explain to me a bit more? Also, what are you referring to when you talk about ‘in visit’ engagement?

    Thanks

  24. 24

    Shahar (and also Nick): I think you have misunderstood my concern as to what aspect of customer engagement web analytics can measure. I am not saying web analytics cannot measure engagement. I am saying that it can only measure one aspect of it, degree, as opposed to another, kind.

    I suggest that even kind can be inferred from degree but this is more of a hypothesis, and that this inference cannot be informed or verified from within web analytics itself. It’s more a matter of common sense and involves a degree of error. Take a look at Nick’s comment (1). The transition of thought he describes ‘if x then y’ is not supported and is not verifiable by means of web analytics. However I explicitly suggest in this post that such inferences can and should be made.

  25. 25

    Shahar \ Eric: One more thing. I think your example is spot on.

    Please help me with this.

    Yes I am sure we would both agree on which user is more engaged. One of my concerns however is with the value of this statement ‘x is more engaged than y’. So what? How engaged is he (x)? I feel that answering this question by saying ‘he is more engaged that y’ is insufficient because it is relative. I don’t believe such relative statements are really useful and operational. I have however pointed to a way (which I did not of course invent myself) that such relative statements can be contextualised and be given a relative, though not arbitrary, reference point that makes them meaningful and useful.

    But why do we both agree? (I assume you consider User B as more engaged) My main reason is because I want to avoid counterfactuals and minimise error. What do I mean by that: we all know, from our own experience as customers and as marketers that it is possible to be a frequent\recent visitor\purchaser that spends a lot of time in a shop\website but who is nevertheless not engaged with the brand in question. This fact leads us to discredit the engagement value of visit metrics (recency\frequency\duration\depth) and emphasise the value of interaction metrics (tagging\commenting\filtering\uploading own content) etc.

    There is however a counterfactual to this case as well. There are highly engaged and loyal customers that have a very high degree of emotional involvement with the brand who are nevertheless passive participants. If we propose a differential weighting of engagement metric components, in this case, according a higher engagement value to interaction as opposed to visit metrics we miss those too.

    The conclusion is that if you do not accord different engagement value to different engagement components you have the following problem. I am going to somewhat change your example to show you what I mean:

    User A goes into You Tube. He browses about 150 pages which brings him deep into the site but he doesn’t watch a single video.

    User B behaves in the way you describe in your comment.

    Remember that in order for our relative ‘x is more engaged that y if and only if x does z and y does >z’ we need to supply a reference point, say our website average of z. That means that both users will come up as having the same degree of engagement. (If not change the 150 pages of depth to 200, or add extremely high frequency, or some extremely high score in one or several of the visit metrics)

    Eric Peterson’s metric, by according equal importance to each of the different metric components of engagement would rate both customers as equally engaged (if ive understood him correctly). Now my big question is ARE THEY?

    This is a question as to whether we should differentially weight different metric components of engagement e.g. accord a different engagement value to interaction vs visit metrics.

    Of course there is no need for a uniform answer. It may be dependent on industry\company as so many other things about engagement are.

    Sorry for the rant. I'm just so perplexed by this issue and would really like to hear what you think.

  26. 26
    Jim Novo says:

    @Theo: I’m not sure much of what has been said here is contradictory, but the importance of different aspects of it depends on the lens you are viewing it through.

    As a Marketer, the part of engagement that interests me most is *dis-engagement* and the process that leads to it. That’s because I can and should take action on dis-engagement, to slow or reverse the process. And the most effective way to do that is to allocate marketing funds to the highest and best use, driving higher ROI.

    Said another way, as long as you are visiting the site (or interacting with some part of the business overall), I really don’t care what you’re doing; you do what you want to do at the pace you want to do it. After all, trying to “force” you to do something differently is not customer centric and could be damaging to the relationship. The ROI on funds spent against people who are already engaged is much lower than the ROI spent on people who are dis-engaging IF (and this is a big IF) you target them before they completely disengage. Once they dis-engage, it’s too late.

    This part of the dis-engagement process is rarely addressed online, because treatment of the relationship is so binary; you’re either a customer or not, you are either visiting or not. And it’s problematic when most of the focus is what is happening on the site, where people are already engaged. Those folks are not the issue, the issue should be the folks who stopped visiting, who are not engaged.

    And they get to this state through a process that you can measure and intervene with. But you need a flag, or a trigger, to alert you that dis-engagement has begun, the most common being Recency or other defection model. Then you know specifically “who” is defecting, who needs attention, which segments are headed for the low value bin.

    Campaigns, a salesperson, product of first purchase, a service rep, content areas, etc. create customers / visitors who are more likely to defect, *relative* to other customers / visitors. The relative piece of it is important, because I am allocating funds; I want to spend where I get the highest ROI – same as when acquiring customers. For every dis-engagement intervention, if I can spend $1 and make $7 on this segment and for $1 I only make $1.50 on this segment, I allocate towards the higher ROI.

    As to “Kind” of dis-engagement, you may have data on this (service records, call center records, products or categories purchased, etc.) or you may not. If you don’t have evidence of Kind, I agree with Avinash that the only way to get to Kind is to survey the dis-engagers. However, I think many folks ignore the opportunity to look for hard data on Kind because it’s easier to do a survey, and that’s a suboptimal approach. It’s too difficult to get the hard data, or they have a systems / data issue and can’t get the hard data. That’s unfortunate but it doesn’t make surveys the “best” way to do it, it makes surveys the only available way to do it.

    As a final thought, the Frequency / Recency framework is intentionally simplistic and flexible, because even though folks might not care right now about that, at some point folks will have to have a universal way of looking at the world that works on any platform, any source, any channel, any technology.

    From a management perspective, measurement frameworks that require “special treatment” for different use cases are suspect, e.g. you can’t use one approach for the web site and another for social media – that just means the model isn’t robust enough. What’s going to happen when looking at multi-channel customers? We have to explain to management that engagement looks like ”this” for the web site but looks like “this” for social media and ”this” for catalogs and “this” for B2B”? If you have 10 different frameworks for measuring engagement, how will you make rational, unbiased decisions where the results can be tracked, interpreted, and acted on in a meaningful way? How will you explain all this engagement stuff to a CEO in a way that demonstrates the value of it, and not have it look like you are torturing the data to fulfill your own needs?

    The Frequency component of this engagement model can be defined any way you want to, and when most folks talk about “engagement”, that’s what they’re doing – they have a customized formula for the interaction part of engagement for different sites and applications – I have absolutely no problem with that, it’s appropriate, it doesn’t break the Frequency / Recency framework. However, many of these models ignore the dis-engagement part of engagement, and that is the most valuable part (for Sales & Marketing), because it’s the most profitable part of the model to take action with.

  27. 27
    Jim Novo says:

    Whoops, forgot to leave a link for those of you who have no idea what the Frequency / Recency model is / came to the engagement party late:

    http://blog.jimnovo.com/2007/04/25/measuring-engagement/

  28. 28

    And so we are in all in this big room of a blog coming and going as we please although we always leave a talking head (or several) behind.

    Many of us take these conversations elsewhere online and offline. A particularly fruitful one has been taking place in Steve Jackson's (Captain Blackbeard) blog which i would like to signpost here.

    http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model/#comment-6723

  29. 29

    OK All,

    Engagement, yes, but for what? Why would you want to evaluate (if not calculate) your visitor engagement if you can't get back to the most "engaged" ones?

    If you've been analyzng web site traffic like me for several years, you know how bored we are now with visits, visitors, page views, and conversion. We got the feeling that we are missing the point! I KNOW I am talking to a minority that has any interest in my site (i.e. the analyzed site); the level of superficial traffic on most sites is just astonishing!

    So, what do I have to better qualify that traffic, since I'm stuck with anonymous data? An Engagement Index at that level would only tell me to what extent I am wasting my marketing budget. However, we can certainly go beyond the GA reports Theo gave as examples, and attribute value to some actions. I mean, subcribing to my newsletter must be a more valuable action than just looking at the product page! But still…

    In the end, Jim Novo's right. All this ends up being an "in visit" evaluation. The real game is in the relationship, and as he points out, whether it's at risk of coming to an end.

    And doing that, my friends, will bring us far beyond the visits/visitor and their cookie problem. This will need to be solved at the multichannel level, with a name appended to the visit.

  30. 30

    Theo – Thanks for your long reply to my comment.

    I think you are right. There could be users that are doing just passive interactions with your brand/web site and are still very much engaged.

    We at NuConomy tried to solve this by allowing you to create as many "engagement" formulas that you want.

    Basically for each formula you can decide what activities will be included and more importantly – what importance weight to give each activity compared to all other activities in the formula.

    Using this method, a user that made a lot of page views can still be considered more engaged than a user that uploaded videos. It all depends on the weight you gave each activity in the formula.

    But I think you were right when you asked – So What?

    Why do I care to know that user A is actually more engaged from user B?

    This brings me to what I believe to b the most important thing about web analytics today (and a point we worked on very hard from day one):

    Web Analytics platforms should not be disconnected from the operational platforms. They should automatically streamline and change the user experience in the site.

    If we know that user A is more engaged, we probably want to give him different messaging while he browse the site.

    We can even decide to offer more help to user B that we know that is more passive and encourage him to become more active.

    Another thing that we try to do in our platform is to correlate this engagement metric to different activites. for example, give you answers to questions such as "Does more engaged users tend to click more on ads or purchase more items?(if this is the way you monitize your site)

    Or even ell you in waht king of activities you want your users to be engaged in order to 'push" them to do other activities (like clicking on ads, purchasing items, etc).

  31. 31

    Jim, I think you right in a lot of your points. It is not possible to compare engagement in a social network to engagement in a online shop.

    Again, I will give an example of how we decided to solve this (some of it still in the works).

    I tend to be practical when looking at web analytics. And the most practical way is of course money. Revenues. ROI.

    Because we calculate not just the relationships between different activities (such as comments, rating, page views, posts, etc…) but also the strength of affect each activity has on the other, we can actually tell you for example how much dollars each comment is worth to you. How much each video uploaded is worth to your business. How much each one of your users is worth to you.

    Now, this information is something that you can compare to other sites, or even to other mediums such as real life shops, tv, etc.

    I know it is not perfect, but I think it brings you a step closer to what you want in the end of the day.

  32. 32

    Dennis\Jacques: A valuable insight using the engagement metric: Breakdown degree of engagement by source or referrer of visitor. How engaged are visitor segments for different digital channels such as paid and natural search, affiliates and display ads? This will help you decide in what source of traffic it is best for you to optimise and allocate your marketing budget.

    As regards the distinction you make between the engagement value of viewing pages, or for that matter any and in aggregate all visit behaviours\metrics, and interaction behaviours\metrics (feedback, subscription, registration etc) please take a look at my long comment to Shahar (26).

    The aim\value of customer engagement? Boost customer loyalty by engaging with your customers and, in turn, increase customer retention.

    Examples of ways to engage with your customers: involve them in new product development or redevelopment, solicit feedback on everything you do, give them the opportunity to rate/tag/filter/comment on your content/products, help them share their product related expertise with the rest of your customers, create community around your brand or participate in communities that relate with your brand etc.

  33. 33

    Theo, your wrote: "engagement metric … will help you decide in what source of traffic it is best for you"

    Good. Finally I see what goal you are trying to accomplish. Now we can discuss Engagement metric within meaningful context.

    So, do you think that "Engagement metric" is more useful for rating incoming traffic than Bounce rate?

  34. 34
    Ned says:

    I still think for most folks spending time on calculating an "Engagement Index" is a waste of time — IMHO, they can derive more ROI by spending that time on improving their websites and customer experience. :-).

    I liked this post for the many points and counter-points it makes on Engagement. Theo, I like your nuances on the "kind" and "Degree" of engagement. I also totally agree with you/Avinash that Engagement just cannot be a "a" metric — it has to be synthesized. To me, Engagement is no different that variables like 'Trust', 'Happiness', 'Attachment' etc. In the Marketing world, we call these latent variables and not directly measureable (but inferred through other measurements). And of course, I also am in favor of the RF[M]L model explained by Jim and mentioned by you.

    However (got to be one, right:-)), there are a few things I don't totally agree with. Here are a few of my counter-points:
    * You mention that engagement boils down a single-concept "one-way relation". I think it has to be a 2-way relationship. You cannot have your customer engage with you unless you first made the effort to engage with the customer (through the appropriate interface, products, price, experience etc.)

    * You mention 'positive' and 'negative' engagement. Theoretically, I agree with you — but on the practical side I think engagement is always positive (this is purely my opinion). I would find it odd if someone says that he married his girlfriend but is [negatively] engaged to his ex :-). Sorry, not a good example but you get my thinking.

    * Now to the Frequency, Recency aspect. I will go out on a limb and say that measuring RFL itself will not tell you much about engagement. In my opinion it has to be anchored down with something (like conversion). For example, I could be going to a restaurant every day for a year and not buy a single menu item (let us say I just like the ambiance:-) ). I would score very high on RF but am I really engaged? The same with the websites — if they really don't convert or do something that benefits you OR something that helps you make the site better, are they engaged?

    * My last comment is on 'Tactical Engagement' vs 'Strategic Engagment'. Jim, I love the Strategic Engagement part and kudos to you on elucidating that concept. But I am not that thrilled with the Tactical Engagement part. To me it is just a misnomer. I would never say I am having a 'relationship' with a girl after the first date :-).

    Net net, I think this was a great post and discussion on a very engaging topic :-).

  35. 35

    Ned:

    * I agree with you it is a two-way relationship. However i don't really know of anyone that measures their organisation's engagement with the customer, that is to say, customer engagement, usually refers to the organisation's measuring and evaluating of the engagement of their target customer's with them. Saying that a similar comment was made earlier on by NickP to whom I responded (comment 14) saying that his comment, like yours made me think twice. In the near future the organisation's engagement with the customer may also be an object of measurement.

    * Though i understand what you are saying there are too many cases where engagement is not positive. Aside all those people negatively engaged with brands who put their efforts continuously to write negative reviews, create sites, participate in social networks, subvert advertising(e.g Tahoe) etc, i believe along with other marketers (such as Huba and McConnell) that the vast majority of customers, who may very well be frequent/recent customers are apathetic to the brands they consume, hence the very low loyalty across industries. From this point of view even if they do buy (your third point) though i would agree says something more than not doing so, is far from being conclusive as regards their degree of engagement.

  36. 36
    Ned says:

    Theo,

    Thanks for the response. Agree with you that measuring organization's engagement is not an easy thing either. On this I was thinking more on aggregate CPRA measures (Conversion, Penetration, Retention, Acquisition) than engagement with a specific customer.

    On the positive/negative part – well, I think we are both in agreement with the concepts, it is the positioning of these concepts where we are bit off.

    * We both agree that there a customer can be positively engaged with a brand/company.

    * On the negative engagement — I think there is 'negative engagmenet' and then what I call 'hostile engagement'. Into negative engagement, I would put negative feedbacks and the like. I think this is good for the company. Also, I believe that a little Negative Engagment (and the firm's proper handling of it) can lead to a stronger Positive Engagement in the future. Also, during a customer's life cycle with a company, they can have both positive and negative engagement – it is the proportion between them that determines the overall behavior.

    And then there is the hostile engagement. It is into this category I would put the overtly negative reviews and the social network backlash from the likes of subvert ads etc.. It is my contention that in many cases, folks involved in hostile engagement are really not your customers. They are either professional critics or just get a kick lambasting brands or are front-ends to lobbyists or the like (going to the Tahoe example, there were customer ads linking Tahoe –> global warming –> length of summer etc.). My opinion is that if you do have a customer that has been hostile for a period of time, you can be pretty sure that the relationship is gone and you have lost them for all practical purposes. That was the basis of my statement that 'From a net net perspective, engagements can only be positive'.

    * You mention Huba and McConnell. I think [I am qualifying my statements as these are purely my thoughts and you can feel free to shoot holes into it or tear it to pieces:-)] there are many sources like them that actually act as moderators to the kinds of engagement. Meaning, customers are finicky creatures. Sometimes, even if their experience has been positive it can change to negative based on what they have read or heard about in the blogs, papers, etc.

    And lastly (and I might be alone in this), I think that we should start by measuring an aggregate customer engagement index for a group or segment of customers and not at an individual customer level. This so because as I mention before a customer can have multiple kinds and degrees of engagement over time and what we want to make sure is that their "Lifetime Engagement Index" always stays positive.

  37. 37
    Ned says:

    Theo, one thing I do want to reiterate. In spite of certain differences in our way of thinking, I did greatly enjoy reading your article:-). You did paint the many aspects of engagement that we should be aware of if we are to make progress in this area.

  38. 38
    Eric T. Peterson says:

    Ned and Theo,

    Perhaps by "negative engagement" you mean "engaged by unsatisfied?" I had a conversation with Larry Freed or ForeSee Results about the relationship between engagement and satisfaction awhile back and (at least for me) the outcome was the realization that the two were distinctly different measures.

    I think this is what is exciting about my operational measure of visitor engagement — over the lifetime of visitor activity it is able to track the ebb and flow of engagement, which can then be juxtaposed against changes in visitor satisfaction (and whatever other metrics you're tracking on your site — conversion, bounce, etc.)

    Neat conversation and nice post Theo!

  39. 39
    Malcolm says:

    Theo – Avinash this is a very valuable post.

    It is even more interesting for me to note the evolution of your perspectives with the feedback in comments. It shows openness and flexibility in thought.

    That is contrasted by entrenched positions of a few here who show no desire to be open to new ideas or flexible in their thinking.

    I commend you for this great resource and for the discussion.

  40. 40

    I didn't grasp the mathematical formula towards the end, which seemed false to me. And the explanation didn't help much, being something of a runon sentence:

    "By inserting relative statements such as ‘x is more engaged than y if and only if x does z and y does >z’ into a continuum that is based on website \ competitor \ industry benchmarks, it is possible to provide a reference point which although relative in itself (historical performance) is sufficiently stable and pertinent to business performance, to provide with useful insights into visitor behaviour and business\campaign success."

    Besides that, I learned a boatload from this post. Interesting to see that largely what at stake here is degree and not kind of engagement.

    On a related note, Scoble had something on this with a great discussion in the comments, imho. Lots of his readers proposed interesting solutions to measuring engagement. The discussion here too has been sweet, so let me respond…

    @ Ned, from my own personal experience being hostile-ly engaged with Google, especially after having previously had positive engagement, I have to contradict this:

    "And then there is the hostile engagement. It is into this category I would put the overtly negative reviews and the social network backlash from the likes of subvert ads etc.. It is my contention that in many cases, folks involved in hostile engagement are really not your customers. They are either professional critics or just get a kick lambasting brands or are front-ends to lobbyists or the like (going"

    First, you're not getting the persona right, imho, or at least not for all the hostile engagees. I'm not a professional critic or lobbyist, but I am passionate about the companies I deal with. For example, I wrote recently about 4 cases of reputation management I was involved in or had some relation to: http://seoroi.com/case-studies/reputation-management-adviso-dreamhost-1and1-network-solutions/

    Second, I rank for 'Google sucks.' And I've bought ads from Google, and will likely continue to, because that's where the volume of search is. I was also very happy to imitate my friends and have Google as my homepage back in the day (before I learned SEO to any serious extent). To an extent I'm contradicting myself here, because I know that some people who need SEO will google that, so it's in my own interests to rank for that (so perhaps your personas weren't so off after all). On the other hand, there are a lot of legitimate criticisms there of things Google does that rankle with me.

  41. 41

    Eric \ Ned \ Gab :

    First off, please excuse my delay in getting back to you.

    My most refined thoughts about a typology of negative engagement has prior to this discussion been defined by the way I see engagement: the subject (customer or non-customer) and degree of engagement (take a look at the table I ve posted here to see how i feel these two variables interact http://agoraplace.wordpress.com/2008/01/04/defining-customer-engagement/). This table shows 2 kinds of negatively engaged customers:

    – negatively engaged customers: These people buy from you but don’t like you . These people who, according to what many marketers believe, and customer defection rates prove, compose the majority of the customers of the majority of organisations, will instantly defect to a competitor as soon as a better product\service, cheaper price, promotion is available. These customers may appear positively engaged from a visits metrics (recency, depth, duration etc) as well as from the fact that they do purchase IF we misunderstand their degree of engagement for kind of engagement. I think this is exactly what Gab is talking about in relation to his relationship with Google.

    – negatively engaged non-customers: These people don’t buy from you and don’t like.
    This is a category of negatively engaged subject I think no company will ever eliminate. In so far a company does not pursue a ‘please all’ mass marketing approach (if that is at all possible nowadays in mature markets and in our heterogenous society) there are bound to be parts of society who will react negatively to these products/services on the exact same grounds the target customers will love those selfsame products/services. The Tahoe is precisely such an example. If you are out to please those people who want a powerful, plate-armoured, gas-guzzling tank-of-a-car, a very real need irrespective of whether these people want to drive it in the city or in the most adverse off-road conditions (I believe psychological and utilitarian needs are equally real – which is not to speak of justification) then you are definitely going to upset those people that are conscious of the mark their footsteps leave on this planet. Similar things happen in fashion, the fast food industry, entertainment etc

    I’m very happy you guys think there is something of value in this post.

  42. 42

    Gab: You are right. There is a mistake in the formula that tries to express that one person is more engaged than another:

    “By inserting relative statements such as ‘x is more engaged than y if and only if x does z and y does >z’ into a continuum that is based on website \ competitor \ industry benchmarks…”

    It should read ‘y does

    Thank you very much for pointing this out!

    However this made me think as to how to rephrase the formula when we have identified and defined the benchmark, hence giving the statement a fixed reference point.

    Defining the symbols:

    z= (number) – this number expresses degree of engagement according to a particular action. Any number greater than this number expresses an above average degree of engagement and suggests the customer is engaged whereas any number lower than this number expresses an above average degree of engagement and suggests the customer is not engaged.

    G= does (behaviour) more than z – slot in whatever behaviour you want in the bracket.

    H= does (behaviour) less than z – slot in whatever behaviour you want in the bracket.

    F = is engaged

    J = is not engaged

    The formula: If (Gx & Hy) then (Fx & Jy)

    In predicate calculus: (Gx & Hy) -> (Fx & Jy)

    This formula says: if x does G and y does H than x is F and y is J.

  43. 43

    Ah, well thanks for clarifying that it wasn't just me being a moron lol.

    Err, the formula's still a bit abstract Theo. Any chance you could get a bit practical? Or just tell me if my paraphrase captures the essential.

    1) You set a behaviour as your proxy for engagement.

    2) You set a level of performing the behaviour as as average engagement. This refers to degree, of course, not kind.

    3) If you perform the behaviour more than said level (step 2), you have a high degree of engagement.

    4) If you're doing less of said behaviour than the average level, you have a low degree of engagement.

    Was that it?

  44. 44

    Oh, and you understood my nuance about being negatively engaged yet still a customer perfectly. The second MSN/Yahoo gains enough share of search to make it worht the effort to set up campaigns there, I'll be shifting dollars there.

  45. 45

    Gab: What you say, I think captures everything, and helps turn 'more than' relative statements into meaningful, is "engaged above\below average" i.e. "is engaged" or "is not engaged" (the degree of engagement or disengaged being a matter of further refining this further).

    In step one it does not need to be a single behaviour. It can also be a composite of equally or differentially weighted behaviours.

    Having defined when engagement and when disengagement begins, it is possible to take action, say by trying to re-engage disengaging customers.

  46. 46
    Devin Ben-Hur says:

    Your formula in the image is wrong.

    http://www.kaushik.net/avinash/wp-content/uploads/2008/01/x-greater-than-y.png

    Reverse the sense or swap the operands of one of the greater-than operators.

    Kapish?

  47. 47
    Raghu Ram says:

    I Only wish i could spend a day talking about problems with the Master Avinash Kaushik..!!

    You are simply awesome Sir..!!

  48. 48
    tania says:

    Degree vs Kind such a useful insight, my best take away is your point Avinash, of explaining the difference to the Hippos at the start of the meeting leading to the question "how can we measure kind"…such a great opener to motivate for Primary research in the form of a short survey…I get such resistance when I suggest these

Trackbacks

  1. […] Avinash Kaushik, autor jednego z moich ulubionych blogów, napisa? ostatnio o roli narz?dzi analitycznych w pomiarze zaanga?owania u?ytkowników online. Problem jest dosy? g??boki i jednocze?nie aktualny. Przecie? o zaanga?owaniu w serwisy spo?eczno?ciowe mówi si? powszechnie, a charakter spo?eczno?ciowy ma mie? ju? wszystko, tak?e sklepy czy witryny informacyjne. Co maj? robi? u?ytkownicy? Anga?owa? si?! I to w?a?nie przychodzi nam zmierzy?. […]

  2. […]
    Occam’s Razor by Avinash Kaushik

    Measuring Online Engagement: What Role Does Web Analytics Play?
    […]

  3. […] has been a tremendous amount of discussion around the Engagement topic that started here and fragmented into a bunch of chunks and related topics, including here, here, and here.  I have […]

  4. […] place to start.  Depending on your business model, you should probably also take a look at what Theo proposes in terms of Kind and Degree for survey work once you have dis-Engagement behavior as a trigger for the […]

  5. […] Measuring Online Engagement: What Role Does Web Analytics Play? | Occam’s Razor by Avinash Kaushik (tags: analytics engagement) […]

  6. […] De vraag is natuurlijk of een vaag concept als engagement gemeten kan worden (lees een reactie van –Rene hier. Helemaal eens natuurlijk). Gelukkig ben ik niet de enige die die vraag stelt (heel goede post trouwens!) en is ook Nuconomy niet te beroerd om die vraag aan de orde te stellen. Aan de screenshots te zien wordt engagement onder meer gemeten door te kijken naar dingen als unieke gebruikers, bezoeken, gemiddelde tijd besteed op de site, bounce hits, page hits, talkbacks, aantal foto’s en video’s geupload, rss subscripties, friend requests, ratings, aantal klikken op advertenties, aankopen, posts en reacties. […]

  7. […] Unter anderem durch die (lohnenswerte) zeitliche Belastung, die die Mit-Organisation des BarCamps erfordert hatte, hatte ich kaum Zeit gehabt, mich auf die Session vorzubereiten. Als völlig unbeschriebenes Blatt beim Thema Community-Forschung versuchte ich, den Input aus der Session und aus meiner einzigen Vorbereitungslektüre fortlaufend in Gedanken zu strukturieren. Vielleicht hörte der ein oder andere Session-Teilnehmer die Zahnräder in meinem Oberstübchen mahlen – heraus kam ein bescheidenes, nach der Session hastig auf Papier gekritzeltes Schema, das ich morgen versuche, hier im Blog abzubilden. Erst mal schlafen gehen… […]

  8. […] I spent all of Monday evening skipping back and forth between the NHL trade deadline and an insightful article by Theo Papadakis on Occam’s Razor about engagement profiling and metrics. He states that while visitor engagement is important for marketers, web analytics can only provide relative degrees of engagement (and even those can’t be proven to be positive or negative) […]

  9. […] I have suggested before, I think that it is very useful to classify the ways in which customers engage along two dimensions: kind (positive – negative) and degree (high, passionate – low, apathy). In order for the scale/continuum of degrees of engagement to make sense it is necessary (read why this is necessary here) to first identify what the average degree of engagement is. I will take positive engagement as an example. […]

  10. […] Kaushik, pisarz jednego z moich ulubionych blogów, napisa? ostatnim razem o roli narz?dzi analitycznych w pomiarze zaanga?owania u?ytkowników online. Trudno?? jest ca?kiem przenikliwy i jednocze?nie dotychczasowy. To? […]

  11. […] Zugleich fehlen nach wie vor grundlegende Informationen, zum Beispiel darüber, wie groß die deutschsprachige Blogosphäre überhaupt ist (unter Berücksichtigung von Serien- und Multibloggern sowie Weblogvakanzen), wie relevant die dort diskutierten Themen für massenmediale Öffentlichkeiten sind (die Annahmen reichen von einer selbstreferentiellen Copy-and-Paste-Kultur bis hin zum Nachfolger der Printzeitungen) und vor allem auch: von wem die Millionen von Einträgen überhaupt gelesen werden. Wie sieht die soziokulturelle und demographische Struktur der Blogleser aus und wie verändert sich diese im Lauf der Zeit? Wie groß ist das Engagement im Neuen Netz tatsächlich? Kurz: der Forschungsbedarf ist nicht zu übersehen. […]

  12. […] If you need a refreshing reminder about making things clearer for the rest of your company, and particularly more senior management, bosses, and CEOs, Avinash Kaushik has some good posts on Occam’s Razor which can feel like they pour a bit of cold water on the evangelical aspects of community and social media – but actually really help clarify the most useful methods of making things simple and effective – rather than relying on enthusiasm, buzzwords, and what it’s easy to assume is the inescapable logic of enagaging communities. Particularly this one, and this one! […]

  13. […]
    Medir el engagement de los usuarios que nos visitan es el tema de moda entre los expertos en analítica web, dada la importancia que esta tomando la web social frente a la web de página vista por visita.

    Medir el engagement sin embargo es difícil porque a diferencia de click o la página vista, que suceden en nuestro site, esto otro ocurre en el usuario, en sus sentimientos y actitudes.
    […]

  14. […] However, in a short term perspective, such as during a visit, engagement and satisfaction are not correlated. They are only related in the sense that satisfaction presupposes engagement. As such, it could be argued that respondents, who do not engage at all (e.g. those who bounce), should be disregarded entirely when calculating satisfaction scores. Given that such respondents have almost no experience with the website their “evaluation” of it must be considered unreliable. By the same token, it could be argued that highly engaged respondents, who still express dissatisfaction, should be given more weight insofar as their evaluations are more reliable. (See also the discussion on engagements in “Measuring Online Engagement: What Role Does Web Analytics Play?” and ”Responding to Geertz, Papadakis and others.”) […]

  15. […] The less interactive a site becomes the more likely users are to click away and do something else. Latency is the mother of interactivity. Though it’s possible through various UI techniques to make pages subjectively feel faster, slow sites generally lead to higher customer defection rates, which lead to lower conversation rates, which results in lower sales. Yet for some reason latency isn’t a topic talked a lot about for web apps. We talk a lot about about building high-capacity sites, but very little about how to build low-latency sites. We apparently do so at the expense of our immortal bottom line. […]

  16. […] Measuring Online Engagement: What Role Does Web Analytics Play? on Kaushik.net. Here Avinash is telling about the quest of measuring Online Engagement and also outlined a four step process you should follow when it comes to trying to measure engagement. […]

  17. […]
    Parmi les rares qui ont pu réussir l'adaptation, nous trouvons le NYT. En effet, ils ont mis en place une équipe dédiée Web analytics. A leur tête, nous trouvons James G Robinson qui est en charge d'élaborer périodiquement des analyses et des rapports sur les visiteurs des sites Web du NYT destinées au département des ventes. Ils utilisent plusieurs métriques, mais le plus important reste « le temps consacré » (à lire un article), comme indicateur clé pour choisir la pub à placer sur la page. Cet indicateur couplé avec l'indicateur de l'interactivité constitue le métrique le plus en vogue en ce moment et qui s'appelle « l'engagement ».
    […]

  18. […]
    Parmi les rares qui ont pu réussir l'adaptation, nous trouvons le NYT. En effet, ils ont mis en place une équipe dédiée Web analytics. A leur tête, nous trouvons James G Robinson qui est en charge d'élaborer périodiquement des analyses et des rapports sur les visiteurs des sites Web du NYT destinées au département des ventes. Ils utilisent plusieurs métrique, mais le plus important reste « le temps consacré » (à lire un article), comme indicateur clé pour choisir la pub à placer sur la page. Cet indicateur couplé avec l'indicateur de l'interactivité constitue le métrique le plus en vogue en ce moment et qui s'appelle « l'engagement ».
    […]

  19. […] Q4: Do you think social media engagement analytics are valid? RE: Avinash Kaushik's posts on Measuring Online Engagement: What Role Does Web Analytics Play? http://bit.ly/d4dETE and Social Media Analytics: Twitter: Quantitative & Qualitative Metrics […]

  20. […]
    What if we measured our PPC team on engagement metrics.   This means that our PPC team isn’t focused on CTR’s for single visitors, but rather attracting a high number of people who will engage with our content multiple times.  Analytics thought leader Avinash Kaushik has interesting models for this.
    […]

  21. […]
    Off-site analytics measures a website’s potential audience and visibility. Learn How to Track Off-Site Clicks as Goals in Google Analytics. Ask yourself : What actions do you want to track? How are you using your analytics to improve your content or track the ROI of that new ad campaign? What Role Does Web Analytics Play in measuring your online engagement?
    […]

  22. […]
    Q&A sites and animal hat-related sites generate more sales than Facebook, which leads to two decisions: Participate in more Q&A sites and get more links from animal hat-related sites. Set up campaign tracking for these. Facebook has a big audience. Diagnose our presence there and refine our measures of success there. Facebook may be a better source for engagement.
    […]

  23. […]
    Engagement is subjective and web analytics tools are inherently unable to measure the kind, positive or negative, of engagement and are left to only measure the degree of engagement. Measuring the degree of engagement is going to be unique to the experience of the site. Maybe it’s the amount of contest submissions, tweets, comments, video plays, likes, it all depends. There are a few standbys however: loyalty, recency, length of visit and depth of visit (all located under Audience > Behavior in Google Analytics. With all of these metrics see what the site average is and then use that as a baseline to achieve against.
    […]

  24. […]
    In our quest for dependable metrics, Avinash Kaushik’s views on engagement caused us to pause and rewrite the blog title. Avinash states measuring engagement is fatally flawed. He argues that we rely too heavily on black and white numbers to infer how customers feel about us. Yet, there are a variety of personal factors we cannot measure influencing our customers’ feelings.
    […]

Add your Perspective

*