Slip-ups in Social Media Analytics

Inconsistent results in social media analytics research can have huge implications for the fields of epidemiology, statistics, and machine learning: Many of these data are used to inform and justify decisions and investments among the public and in industry and government.

For behavioral scientists, the increased penetration of social media has seemed as an extraordinary opportunity to capture and analyze numerous amounts of information to predict human behavior.

Specifically, more recently the ‘consumer brand engagement’ (CBE) concept has been postulated to more comprehensively reflect the nature of consumers’ particular interactive brand relationships, relative to traditional concepts, including ‘involvement.’

However, according to an article published by journal Science, Ruths and Pfeffer of Carnegie Mellon’s Institute for Software Research pinpoint several issues involved in using social media data sets:

  • “Different social media platforms attract different users – Pinterest, for example, is dominated by females aged 25-34 — yet researchers rarely correct for the distorted picture these populations can produce;
  • Publicly available data feeds used in social media research don’t always provide an accurate representation of the platform’s overall data — and researchers are generally in the dark about when and how social media providers filter their data streams;
  • The design of social media platforms can dictate how users behave and, therefore, what behavior can be measured. For instance, on Facebook the absence of a ‘dislike’ button makes negative responses to content harder to detect than positive ‘likes’;
  • Large numbers of spammers and bots, which masquerade as normal users on social media, get mistakenly incorporated into many measurements and predictions of human behavior;
  • Researchers often report results for groups of easy-to-classify users, topics, and events, making new methods seem more accurate than they actually are. For instance, efforts to infer political orientation of Twitter users, using Twitter analytics achieve barely 65 percent accuracy for typical users — even though studies (focusing on politically active users) have claimed 90 percent accuracy. Twitter users achieve barely 65 percent accuracy for typical users — even though studies (focusing on politically active users) have claimed 90 percent accuracy.”

The implication is for researchers and behavioral scientists is to be more aware of what they’re actually analyzing in terms of social media data. Studies have been predominantly exploratory in nature, thus generating a lack of empirical research in this area to date. Scholarly implications may focus on closing and addressing the literature gap.


Director of Organizational Development