How accurate is Klout?

Over the past few months there has been a lot of talk about Klout (www.klout.com) rapidly becoming the default system for measuring Twitter influence, with the company selling promo programmes allowing brands to target top people on Twitter.

However, these is some discussion about its accuracy.

Blogger Shel Israel (thanks @jas for pointing me to it)  has a piece up questioning some of the scores.

For example, he points out that A-List blogger Robert Scoble with a score of 83 is virtually tied with US President Barack Obama on 86 (apparently the average score is 11).  P Morgan Brown has added to this by posting “brands beware, my Klout score is a farce.”

The essence of Brown’s point is that he has a higher score than a lot of well known authors – they may tweet less than him and have a lower score but the reality, which can’t be measured by a computer, is that they have more actual influence than he does.

Our view at Rabbit is that as far as influence measurement systems go, Klout is actually one of the best around.  Indeed its even written into some of our contracts as a KPI.

We can also see a clear difference when brands and clients are talked about more on Twitter – for example client (London) Gatwick Airport saw its score increase from 53 to 68 over the past week following disruptions caused by snow, and the use of Twitter as a passenger information tool.

However, Brown in particular makes a good point.  Computer generated algorithms sometimes can’t take account of how things are in the real world and any evaluation and sentiment scoring system still requires a certain amount of human intervention and sense checking.

Update - this post by Adriaan Pelzer of RAAK is definitely worth reading.  Essentially Adriaan created a number of bots to try and artificially create a high Klout score and succeeded.  The post also contains a comment from the Klout boss Joe Fernandez who pledges to cut out that particular problem.

Clearly as Adriaan and others have shown, Klout like almost every other evaluation system has its problems and the social media sentiment industry is still very much in its infancy.     How we’ll continue to use it as a benchmark – to take influence snap-shots in time.   So not withstanding the fact that its possible to cheat, if you have a brand that has been in a certain place on a certain day, then all things being equal you can still see what impact your work has had depending if the score goes up or down.

Enhanced by Zemanta

3 comments

  1. Any ‘accuracy’ achieved by algorithms will last only till the algorithm which will game the original is available. Preventing such automated Klout building algorithms means that the derivation formula behind the original algorithm is as non-transparent as possible…

  2. Hi Dirk

    The question of How Accurate Anything is, really depends on having an understanding of what you are measuring.

    We can only know how accurate a ruler is because we have an agreement of how long a metre should be, which starts with an agreed standard (see here http://physics.nist.gov/cuu/Units/meter.html )
    Once you have that you can start to test the validity or accuracy of different scoring methods (like tape measures, rabbit hops, ruler, laser rangefinders, beard seconds).

    So the question is what is the digital authority of say @BarackObama, vs @Jess vs @ev, is that without knowing where or what the values for these accounts should be on any dimension, you have no way of working out whether something is accurate or not.

    It’s like IQ tests. Do they measure intelligence? Probably not. Do they accurately measure the performance on that IQ test, yes.

    It’s suspicious that you can move the dial on a metric related to influence or authority over the matter of a few days. Surely authority is something more persistent that is earnt not just gifted in a matter of moments?

Comments are closed.