gotcha

Sentimental Gotchas

  |  February 21, 2012   |  Comments   |  

When looking at sentiment results, it's imperative you do not use the metrics at face value. Here's why.

In measuring social media, the current accepted standard that we are using right now is brand sentiment - in other words, the percentage of tweets, posts, and forum discussions that are skewed either positively or negatively toward the brand in question.

What happens in systems like this is that social media analysis providers like Brandtology, Radian 6, and Meltwater would take a feed/crawl of social media sites, use automated semantic analysis to measure the positivity or the negativity of the keywords, and assign a rating to the post accordingly. The posts are then aggregated and a sentiment rating is assigned to the brand overall - for instance, if 50 percent of the posts are deemed to be positive, and 50 percent negative, the sentiment for the brand would be 50 percent overall - quite neutral.

Now, the process described above is obviously a very simplistic view. And the actual process itself is obviously less straightforward, but the end result will always be clear, concise, and simple enough for C-levels to understand; the basic premise being whether the social media sphere is talking good things or bad things about the brand on any particular day.

Gotcha: Semantics, Semantics, Semantics

However, when looking over sentiment results, it is imperative that you do not use the metrics at face value. For one thing, most semantic analysis tools, advanced as they are, still only scratch the surface of the human capacity for linguistic mangling... something that is exacerbated in the social media world.

This obviously would lead to a lot of skew in the sentiment results, and where possible, you should go through all the posts to reassign the sentiment of the posts where a machine might not have assigned the sentiment rating correctly (most solutions out there will allow you to do that).

Gotcha: Internationalization

Internationalization is also another issue. There is the obvious issue of non-English languages that the current systems have to support... but even for English language posts, tweets, and status updates there are dramatic differences in the way English is written colloquially on the Internet across different markets that could drastically change the sentiment of a post.

Further muddying the waters is the fact that in many Asian cultures, even if English is the de-facto language, it is mixed with local language in a way that makes it hard for systems to parse out the actual sentiment of the post (for instance, a tweet from the Philippines could switch from English to Tagalog effortlessly within the space of 140 characters; Singlish is another renowned linguistic hybrid).

Gotcha: Abbreviations

Case in point, we were doing a social media scan for a big bank in Indonesia for which the name was an abbreviation. As it turns out, that particular abbreviation was also the same abbreviation for a typical work in Bahasa, and the signal-versus-noise ratio increased exponentially, making it harder to get good insights out of the social media scans.

Conclusion

Now, you might be asking what the point is in pointing out all these gotchas of sentiment analysis. What we have found is that sometimes, people put too much faith in sentiment ratings as the be all and end all of social media analysis where it is actually just barely scratching the surface. What we would need to do is the following:
  1. Use sentiment analysis as the start of your analysis, not the end point. Scrub through the results; rejig the ratings manually if possible. It is the insights that are important for action, not the rating itself.
  2. Always, always be aware of the gotchas when looking through the results. This will skew and enable you to get more value out of your analysis.
And most importantly, remember that social media analysis is just measurement at the end of the day - the utmost thing is action. Measurement is used to guide action, and without immediate action especially in the fickle world of social media, your measurement tools will just be another white elephant in your repertoire.

Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!

ABOUT THE AUTHOR

Adrian Lee

Adrian is the chief of digital marketing and technology in Yolk, a Grey Group company, one of Asia's leading interactive and digital media agencies with over 40 employees headquartered in Singapore. Adrian joined Yolk in 2005 and helped shape the vision towards a company where creative and technology is inexplicably linked to serve the higher purpose of marketing. With this approach, Yolk managed to secure regional accounts such as Microsoft, Cibavision, and Canon. Adrian has 12 years of experience in the digital industry with parts of those years spent in Microsoft being in charge of MSN Search, Portal, and advertising platforms, overseeing the expansion of MSN portal from a single market (Singapore) to five markets across Southeast Asia, part of the team that piloted Microsoft adCentre in Singapore and won "Global Product Manager of the Year" at Microsoft in 2004. His technological background is well complemented with his five years experience in advertising and publishing industry. Technology solutions, which Adrian creates, always serve the purpose of his clients in bridging the latest technologies with marketing strategies to boost their campaigns to their fullest potential. When not knee deep in technology, he produces electronic music under various monikers.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

    Information currently unavailable

Jobs

    • Lead Generation Specialist
      Lead Generation Specialist (The Oxford Club) - BaltimoreThe Oxford Club is seeking a talented writer/marketer to join our growing email lead-generation...
    • Health Marketing Editor
      Health Marketing Editor (Agora Inc.) - BaltimoreCome flex your intellectual muscle as part of Agora, Inc’s (http://agora-inc.com/) legal team...
    • Technical Business Analyst
      Technical Business Analyst (OmniVista Health) - BaltimoreOmniVista Health is looking to add a Technical Business Analyst to our expanding team...