Twitter is amongst new media channels that are challenging how we communicate, with whom we communicate and perhaps most fundamentally how we (Marketers) influence people.
Sadly execution and analysis of these new social media channels has been hobbled by old world thinking. When it comes to marketing because of the old world thinking from the worlds of sTelevision and Magazines, and when it comes to measurement because of the world of traditional web analytics.
These new channels, Twitter and Facebook and YouTube and Tumblr and, yes, even blogs, are very distinct customer / participant experiences. Stale marketing or measurement thinking applied to them results in terribly sub optimal results for all involved.
So in this post my hope is to share with you what is unique about measuring one such channel, Twitter. The blog post is also sprinkled with my own words of folksy wisdom as to how you should use the channel for maximum impact.
My new book Web Analytics 2.0 covers social media measurement, but I am going to cover something very different in this post.
First: An Ode to New Thinking:
One common thing between the all tools in this post is that they were built by "outsiders".
One of the things I love and adore about Twitter (besides all that connection and conversation) is how its open API has lit a fierce fire of innovation when it comes to analytics. Anyone and their brother and ma-in-law can develop a tool, and they have! Much to the benefit of the rest of us.
Perhaps the most beneficial thing to me is how much out of the box innovation this has brought.
For example just look at traditional web analytics tools, there is absolutely no fresh thinking when it comes to Social Media Measurement. Their constant focus is on "let's figure out how to collect and report ever more data and not bother with a truly immersive understanding of these channels and what makes them unique". That mental model is, sadly, extremely clear in the metrics and analysis they provide with "twitter integrations".
While there is some stale thinking in the new twitter tools, most of them have a lot of fresh thinking from people untainted by Omniture or CoreMetrics or WebTrends or, ok ok ok, Google Analytics.
I consider this massive proliferation of new thinking to be a gift from God.
To all of you developers who are toiling out there, you have my love and gratitude.
In this post four twitter analysis tools that while not yet fully developed show sweet signs of:
1. Truly understanding the medium and uniqueness of the channel and
2. Are not just reporting "hits", rather coming up with clever metrics.
Quantitative Metrics / Analyses.
Most twitter analytics tools just do data puking. They find numbers that can be computed and then proceed to puke at you as many as they can find, with wonton disregard of value being provided or outcomes being measured.
Here is one of the mild ones:
You must pause and think: So what is this saying? What action can I take?
Always, always, always ask that question when faced with tools that simply puke data out at you (twitter or Google Analytics or whatever).
But as I mentioned at the start of the post one of the beauty of twitter's open API is that there are a few pockets of truly innovative thinking.
Here are some that I humbly believe look promising. . . .
Klout. Twitter Analytics.
Klout is a wonderful little tool that measures Klout Score, a proxy for "influence":
It is easy to understand the market demand to boil things down to one number, but this is perhaps the least useful thing in Klout.
While on the surface they might seem useful, I am always suspicious of compound metrics. They can be subjective, inapplicable to many and efficiently hide the insights you need to understand what actions to take. [See more here for Compound Metrics: Four Not Useful KPI Measurement Techniques]
Mercifully there is so much more to Klout than that.
Klout measures a bunch of lovely metrics, specifically applicable to Twitter, that are grouped into four buckets: Reach, Demand, Engagement (!!) :), Velocity.
There are two lovely things about these computations.
1. Joe and team have wonderfully avoided the temptation make these compound metrics (as in Reach = Followers / Total Retweets * Friends + Pixie Dust). The factors used are laid out as individual metrics making it easy for you understand the data and pick metrics that work for you.
2. (My favorite) The metric definitions are not "crap". This seems like such a low bar to meet, sadly far too often metrics out there (not just for twitter) are just plain shoddy.
For example here are some clean definitions from Klout:
* How diverse is the group that @ messages you?
* Are you broadcasting or participating in conversation?
* How likely are you to be retweeted?
* Do a lot of people retweet you or is it always the same few followers?
* Are your tweets interesting and informative enough to build an audience?
* How far has your content been spread across Twitter?
* Are people adding you to lists and are those lists being followed?
When I use Klout I simply pick the metrics that are most important to my own twitter strategy.
I would suggest that this is very very very very important, pick what is right for you rather then following a lemmings like strategy of "I am going to use metrics Y & Z that someone recommends".
Here's an example: I don't care about Follower/Follow Ratio. I think it is disingenuous to follow everyone who follows you just for appearances sake when you have no intention of reading what they all say. Why be fake?
As you might have read in the new book I like "Message Amplification" in Social Media, and hence I do care a lot about Total Retweets.
[Sidebar: my favorite twitter metric is: # Of Retweets Per Thousand Followers, it's a measure of efficiency and value provided and people voting with their clicks, all rolled into one!]
I care a lot about Follower Retweet % ("Do a lot of people retweet you or is it always the same few followers?") because I want to appeal to more people than my mom, dad, and best friend!
One of the biggest mistake companies and brands make about Twitter is that they think it is one more "shout channel" like TV and Radio and Magazine ads or Press Releases. Twitter is not that. Twitter is a "conversation channel", a place where you can find the audience relevant to you (and your company and products and services and jihad) and engage in a conversation with them. It is not pitching, it is enriching the value of the ecosystem by participating.
Hence I like the metric Messages Per Outbound Message , as a primitive measure of the fact that you are participating in a conversation and not just yelling.
With Klout I can choose the metrics that best reflect my personal twitter strategy, I can easily find them and I can monitor my progress (using a handy dandy graph) and ensure my strategy is a success.
Your strategy might be different. Walk up to the buffet and pick the metrics that will help you best measure your own success.
Contest: Notice the metrics I have deliberately ignored: # of followers, # of retweets, @ mention count etc. Can you guess why? :) The person with the best guess gets a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]
At the bottom of the Stats tab Klout also includes a handy dandy Analysis table with trend indicators. . . .
As an Analyst it might be of some value to look at the trend pointers at the bottom (clearly I am doomed!), it might be cute to put this into a PowerPoint slide for the HiPPO's who might like the Chinese fortune cookie messages for each metric group.
Ok, ok, I am just teasing the Klout team, I know it is very hard to "wordify" and programmatically make valuable recommendations. :)
GraphEdge. Twitter Analytics.
The reason I believe GraphEdge is interesting is that it has a set of really cute metrics that help bring a different perspective to measuring Twitter.
If want to contrast the difference in thinking applied compare some of the metrics below with, for example, what Omniture is touting with its Twitter "integration". The difference between the old web analytics thinking and a new person's could not be more clear.
[Allow me to rush and add that while Omniture has a hack to bring some twitter data into Site Catalyst to do something, Google Analytics has nothing. Not even something that is not useful. So perhaps GA stinks even more.]
Here are some, IMHO, differentiated metrics. . . .
If you have spent any time on Twitter you know that spam accounts are a problem so it is very nice that the first thing you see in GraphEdge is not a follower graph but rather their attempt at telling you how many legitimate followers you have (and the trend over time, cropped out in the image below). . .
To identify "legitimate" they use the following filters, direct quote:
Any of your followers who are following more than 2,000 people are considered not-Legitimate… they’re probably not really monitoring your feed, so we don't count them as "Legitimate".
Users who have been suspended by Twitter can’t read your tweets (and probably weren’t interested in the first place!). We don’t consider these Legitimate Followers.
It is ok to argue with their filters, but it's a fine start and I think good enough.
Klout also measures something called "Reach", which is also their way of identify if you've got people or bots following you.
In my days at DirecTV one of the metrics that the company was obsessed with atleast then and rightly so, was Churn Rate. It reflected the value of not just going after new customers but doing all that was possible to take care and love the customers we already had. Makes sense?
So I have always had that obsession with tracking Churn, simply to try and understand why people quit. The hope is if I can understand why then I can do something to fix the problem.
[By now I am sure you get the feeling that I am treating twitter analysis like I would business analysis. Twitter is my brand channel and I take this very seriously. It is perfectly ok to use Twitter to tell people where you are and what you are doing and not care about analysis.]
I have not really found any decent tool to track unfollows in twitter (yes I have tried the normal ones and they are either flaky or just outright stink). Hence I was happy to have this in GraphEdge. . . .
Actually ouch! 291 un-follows!! So sad.
Atleast now I know.
It is nice to have the over all trends on the right in the above image, as well as for the period you choose smarter metrics like growth rate.
GraphEdge will also show a list of your new followers and un-follows (so you can send them bad vibes! Kidding, Kidding. :).
[Qwitter was one of the first tools I used to track unfollows, sadly it does not work any more, and it had a great feature: It would try to guess and report on which tweet resulted in the un-follows. Nice.]
And here's what we were on the quest of. . .
I will admit to not being charmed by having three different lines above, they clutter the left and the right, and get in the way of understanding the data. But you I suppose you can learn to ignore two of them.
Here's the definition from GraphEdge for Churn:
The number of removals (un-follows) over the average size of the existing base (followers) during the period measured:
Drops / (Current Followers – ((Adds – Drops) / 2))
As always look at the trends, the longer term the better. And remember that history is littered with companies that were growing just fine but they still died a painful death because of Churn Rate.
Slightly along the same lines GraphEdge has a metric called Loyalty. At the moment I think it is too limited in what it actually measures, and it only starts measuring once you join GraphEdge. But there is kernel of promise in the metric, keep an eye on it.
Lastly… while most people overestimate their "twitter power" (I can bring you down with a single bad tweet Avinash!) I think a few people also underestimate their reach, if they participate in twitter in the right way.
Looking at the second level report can give you a feel for your network size.
Followers' Friends is an "incestuous" number, it shows all your followers and the people they are following. If you have ten followers and they all follow each other that's 100 Followers' Friends. Feel free to be proud of this number, but then promptly ignore it.
Unique Names is are the unique twitter account id's in the network, less the "illegitimate" ones. Think of this as something close to, but not the same as, the Unique Visitor concept in web analytics.
This is a useful number.
Think of it this way: If you say something of incredibly profound :) . . .
. . .and a whole lot of other people who follow you think that and retweet it then you have a theoretical capability to reach 1.2 mil people (Unique Names in your Second Level network).
Now the reality is that that will rarely happen, if ever, but in our profoundly hyper connected world Unique Names is a good number to keep on your horizon.
Remember success in twitter comes from participating in the conversation and giving something of value, not by running "social media campaigns". If you don't internalize that be ready for a reality where both your Followers, and Second Level Network size, to be small potatoes.
Qualitative Metrics / Analyses.
Now let's tackle the much much harder analysis to do in any filed, analyze the data from a qualitative perspective.
I have given up on "Sentiment Analysis". Well atleast for now. Everyone over-promises and massively under-delivers.
On paper it seems like such a great thing to want to have, this is a social / conversation medium after all. But most tools I have had the good fortune to try are simply either glorified versions of Google Alerts even if they promise you buzz metrics and the moon.
Now sentiment analysis is a very hard problem to solve. For example I just analyzed my account using an expensive "social media sentiment buzz analysis tool" and it marked this tweet from today as Negative:
avinashkaushik: There in nothing quite like AC power in your seat for a 10 hour flight. Oh and 20 hours of pending work to do.
Perhaps the tool does not find my dry wit as funny as I do, but it's hardly "negative"!
With all that context I think TweetPsych holds a lot of promise.
Tweet Psych uses the Linguistic Inquiry and Word Count (LIWC) method and the Regressive Imagery Dictionary (RID) method to build a psychological profile of a person based on the content of their last 1,000 tweets.
Dan Zarrella, founder, says: "I think the possibilities of a system like this are enormous, from matching like-minded users to identifying users that exhibit certain useful or desirable traits."
I am not sure I understand perfectly how it works (I need to send this to Joseph Carrabis!) but the analytical techniques looks very promising. . .
Hmm… interesting. I do like talking about "learning, thinking, knowing etc"! :)
As always rather then looking at my data in isolation I compare / contrast it with my friend who is a web analytics twitterer. . .
Now I understand a lot better how I am doing and how he is doing.
Remember there is nothing wrong or right here, we are both just very different people with different twitter strategy and what Tweet Psych's linguistic analysis algorithms helps us understand if our psychological profiles are aligned with our twitter goals.
Tweet Psych also provides you with Primordial, Conceptual and Emotional Content analysis, here's mine. . .
Use this type of analysis to understand at a deep level what attributes are being associated with your brand, and if they are reflective of the goals that you set for yourself.
Content Visualization with Stream Graphs:
Stream graphs can be very good at visualizing data, content specifically. Twitter StreamGraphs is delightful for:
1. its visualization of highly associative words with the word you are querying and
2. viewing streams (tweets) for any associative word (hence sweet filtering, something so darn hard to do with twitter content)
Here's today's view of the data for my account (searching for @avinashkaushik). . .
Please click on the image for a higher resolution image.
You can choose the associative word stream that you are interested in most, click on it and at the bottom you can see the tweets. The size of the stream and shows you strength.
Once you choose the stream you can also click on the dates on the x-axis to filter down to the tweets for that particular stream for that particular date.
The stream graph for avinashkaushik would be different, as it looks for mentions. . . .
Please click on the image for a higher resolution image.
It is analyzing the last 1000 tweets and such a great way for me to understand the content, and filter down and review the relevant tweets easily.
Twitter StreamGraphs helps you visualize content in a very unique way and solves a very important problem to boot.
Parting Words of Wisdom.
I hope if there is one thing I have convinced you of then it is that you need to be a lot more critical when you think of analyzing these new media channels.
It is important to put aside stale (certainly current web analytics) thinking.
It is important to participate in these mediums so that you'll truly appreciate what their real strengths are.
It is important to question metrics that have cute names, dig one step deep, just one single solitary step, to check if the metric definition passes the BS filter.
It is important to choose the metrics that help you measure your unique goals.
Finally it is important to realize there are no short cuts. Be willing to work hard. Be willing to put in the sweat equity. Be willing to try 45 things (tools / metrics / strategies) to find the 3 that work for you.
Missed the contest? Go back and look for the red parenthesis, you'll win a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]
Please share your feedback on this post via comments. Got any other tools that you love and adore? Please share them – with a quick comment on Why you love them. Got a piece of analysis that you think is profound? Please share that with all of us as well.