Losing Klout: Four Questions About Third-Party Influence
ADOTAS – Recently blogger and PhD statistician Alex Braunstein wrote an insightful piece titled “Why Your Klout Score is Meaningless.” In it, he analyzed the Klout Score, one of the most hyped measures of influence today. The article called out Klout for being inconsistent and unpredictable; easily gamed; and not correlated or calibrated against real world measures of value, such as site traffic or sales.
Alex’s thought-provoking article brings up some interesting questions for marketers who leverage influencer marketing to build their brands and drive sales. Influencer marketing requires marketers to carefully identify the individuals and audiences online who can send their brand skyrocketing (or plummeting to earth) based on which content they share, with whom and what they say.
To find out if scores like Klout are indeed a good way to identify super influencers, first you have to ask four pertinent questions.
1. Is that all of the data?
Klout, Peerindex and other third-party social metrics vendors aggregate social data, but do not own or control the inputs. These social data aggregators are built on top of a handful of open social network APIs, mainly Twitter and LinkedIn, and collect data provided by those services.
Given this, it’s not surprising these scores can be an inconsistent and unreliable method of determining influence. If you are only monitoring sharing activity on social networks, you cannot provide meaningful analysis of sharing patterns that show who your true influencers are – because, the truth is, a significant amount of sharing actually occurs outside of the major social sites.
Many influencers pass along your brand’s content via email, blogs, and message boards – but this type of sharing activity is not measured by third-party social metrics vendors.
2. Are third-party influencer scores predictive?
What does it mean in the real world if my Klout score increases by 5 points? Did I become 5 points more influential? The next time I share, will I drive 5 units more reach, referrals or sales?
Braunstein’s analysis suggests there is a dubious correlation between changes in Klout scores and movement in influencer outcomes like reach, referrals, and sales. True influencers, those people who make a measurable and growing impact on bottom-line sales and brand reach, share content far and wide via email, blogs, and forums in addition to Twitter and Facebook.
Putting a score on influence is a far more complex algorithm than just counting followers or retweets; instead, you need to measure web-wide sharing patterns and correctly attribute sales to the influencers who are connected to large numbers of interested users and relevant networks.
3. Can an influence score be gamed?
Anytime you introduce incentives to achieve a certain status or reach a certain score, you will attract people who find clever “exploits” to increase their score. Exhibit A of this effect is the SEO industry. In search, there is a huge economic incentive for individuals to do whatever they can to improve their site’s rankings in natural search results.
As a consequence, natural search results are too often cluttered with poor quality content produced by SEO “black hats” in the form of link and content farms. Google and Bing spend a tremendous amount of time and money fighting the “search spam” problem to their credit, but it is a never-ending battle.
By putting a system in place to reward “influencers” that depends on inputs that can be easily gamed (e.g., the number of followers I have on Twitter, the frequence of retweets, etc.), people will focus on gaming their scores rather than doing things that result in effective, lasting influence. Outside of celebrities, how do you know if a person has a high score because they’re truly influential or because they’re really good at increasing their Klout score?
4. Who (or what) is influential?
Justin Bieber has over 10 million followers on Twitter and his Klout score is 100. Professional snowboarder Shaun White has just under 500,000 followers and his Klout score is 61.
If you are an avid snowboarder, who are you more likely to follow? Who are you more likely to respond to when they tweet about snowboarding equipment or the Olympics? Now, if you are a marketer of winter sports equipment, which of these two individuals is more likely to reach and influencer your target audience?
The intuitive answer is Shaun White, and yet according to Klout, Justin Bieber is far, far more influential than Shaun White. When an influence score is simply based on popularity and not on the quality of interactions, it loses its meaning.
Social media influence scores are interesting social metrics; they show who is best at getting a large number of people to follow them. So, for that, they do show who is a bigger celebrity. But are they really a useful marketing tool for brands looking to identify the people or audiences who truly spread recognition and boost sales?
To understand who your influencers truly are, marketers need to take the focus off the “score” and go beyond using third-party influence metrics to identify web-wide sharing patterns on social networks, blogs, email and other channels.
This is great analysis.
Let me give youa PeerIndex perspectice.
4. Who our what is influential?
What does influential means? Influential is as you point out category and context dependent.
A user like http://www.peerindex.net/gcluley has a PeerIndex of 66 but a topic PeerIndex in Computer Security of 82.
This tells us that Graham is one of the most well regarded people in the subject of Computer Security. And we have scores in several thousand other salient categories.
The single overall PeerIndex score (in yellow) is designed to be digestible and give you a rounded sense of a persons overall impact or status – but if you are going to use our data to identify patterns of influence, you’ll need topic scores.
3. Can an influence score be gamed?
We like to think of PeerIndex as a rating, although we sometime fall into the vernacular of ‘score’.
Can it be gamed? Well – yes any rating or metric can be optimised – companies move assets off balance sheet or sell assets in order to maintain credit ratings.
However, you can design a system knowing gaming will happen. What we have tried to do is design a system where ‘gaming’ will most likely be the result of good behaviours that would reflect genuine expertise. It’s the ‘walk like a duck, talk like a duck’ phenomenon. Their are structures in the graph you should be able to take advantage of.
We don’t yet have widespread examples of the twitter equivalent of link farms – and we have plans to deal with them.
2. Are third-party influencer scores predictive?
They need to be predictive. Once again, we have built a model of predictiveness in PeerIndex, which is perhaps not totally clear in the consumer site. But not only is our goal to build rich predictive ratings and metrics, we have already show several scores that are good predictors.
The best example of this is topic PI – http://www.peerindex.net/azeem – if you look at my ‘Top Topics’, the scores in Green are Topic PI scores, and they at one level tell you the likelihood that I’ll talk on one of those topics and be listened to or agreed with, relative to other people talking in those topics. And conversely, the expectation that someone who sees a message from me on that subject has that the message will be interesting and relevant.
1. Is that all of the data?
This is a signals basis. And signal width matters. We’ll obviously work hard to improve our signal width. And yes – stuff that happens off line is harder to see, but there are ways of doing what we call ‘offline authority’. We’ll keep you posted.
Thanks for a constructive and thoughtful article
That’s interesting Stuff Ben…
We’ve got some thoughts about all of your points – in fact, we designed our product, mPACT, with a lot of the same concerns in mind. To your points:
1. Is that all of the data? No one is ever going to be able to evaluate how influence is being wielded in private conversations. However, focusing on one’s influence across social networks, AND traditional outlets and blogs provides some good data to judge influence. For example, it’s common to find a reporter who writes for a pub, contributes to a blog, runs a personal blog, tweets and is engaged in online communities. You can’t determine how influential that person – or anyone, for that matter – is unless you look across all the places on the web in which they’re speaking publicly. Furthermore, the sharing activity you mention can be used in influencer scoring if done properly.
2. Are third-party influencer scores predictive? We certainly think so. We look at not just “raw” influence, but also equally weight topicality i.e. what someone is writing about, how frequently and how recently. We believe scores should reflect an Influencer’s propensity to write about the topic you care about and their ability to move a market when they do. When one starts to look at influencer scores not on a generic basis (my overall score is X), but instead to look within each individual market segment or community, you can indeed find the voices most moving these segments, and find predictive patterns for their continued desire and ability to do so. For brand managers, this is particularly important. Influencer identification MUST be done within each individual market segment or community. Otherwise, it’s just generic noise of little use.
3. Can an influence score be gamed? There’s no system that’s completely “un-gameable”. But to game a system that’s focused on topical influence someone would need to write more and more on that topic. Random linking wouldn’t be enough.
4. Who (or what) is influential? You can’t measure influence until you first define a topic/market/area of interest. Popularity doesn’t equal influence. Pick your favorite pop culture superstar for example with a high “score” and see how much impact they have on, say, the healthcare IT marketplace. I’d venture to say none. Influencers have the right mix of reach, authority and topicality, no matter what their outlet, whether they have 2000 followers or 2 million. At mBLAST, we understand this. We look for influential voices within market segments and communities. And because of this, we can help brand managers and other marketing professionals uncover the voices that really matter today to their marketing efforts.
CEO – mBLAST, Inc.
This has turned into a lively discussion about the nature of influence. Ben, Azeem and Gary all seem to agree on several key points. The first of those is that influence, when tied to a specific interest, is a more useful measure. The interest graph is how we at Meteor look at that data. To build an interest graph we pair an influencer with a particular item of content and look at the journey of that content, over multiple generations and across multiple channels to see not only who is influential, but also what is catching interest, where the sharing is taking place, and, importantly, how many conversions are tied to that original influencer. An accurate picture of an influencer’s interest graph gives a marketer valuable information about their social audience.
Everyone also seems to agree that the more broadly we can capture data, the more relevant that measure of influence will be.
It seems that the big difference is a matter of perspective. Most marketers I know view influence from the perspective of the finish line. In my mind, the people who drive the consumption of content are interesting, but the people who really matter are the ones who are able to affect outcomes. From a business perspective, those outcomes equate to bottom line dollars. Instead of bothering with assumptions about who might affect ROI, smart marketers arm themselves with real data to focus on their actual influencers, the ones who have proven their ability to move the needle. Armed with real data, they can better tailor the content they deliver to the real and proven interests of their market and make better decisions about where to spend marketing dollars.
Senior Marketing Manager
Thanks for your thoughtful replies Gary and Azeem. Thanks also to you Laleh for taking the time to comment! It is really good to see your tools address many of the questions highlighted in the piece.
- Pingback from Pamflet romanesc