How Natural Language Processing Helps Uncover Social Media Sentiment

JRehling is former senior software engineer and NLP expert at Meltwater Group, where he was responsible for adding natural language technology to the company’s news and social media monitoring services. He also blogs about NLP.

Every day, people discuss brands tens of thousands of times across social media sites. Companies want a slice of that pie to determine how their audience communicates, and more importantly, to discover important information that drives business.

However, the sheer volume of social conversations makes that difficult. Digital marketers and PR professionals are turning to social media monitoring and CRM tools to help filter that social noise and properly engage with their target audiences.

Can these tools really help detect and analyze how a customer feels about a brand? It’s possible with Natural Language Processing (NLP).

What Is NLP and How Does it Work?

NLP goes by many names — text analytics, data mining, computational linguistics — but the basic principle remains the same. NLP refers to computer systems that process human language in terms of its meaning.

Apart from common word processor operations that treat text like a mere sequence of symbols, NLP considers the hierarchical structure of language: several words make a phrase, several phrases make a sentence and, ultimately, sentences convey ideas. By analyzing language for its meaning, NLP systems have long filled useful roles, such as correcting grammar, converting speech to text and automatically translating between languages.

What’s NLP's Role in Social Media?

NLP can analyze language patterns to understand text. One of the most compelling ways NLP offers valuable intelligence is by tracking sentiment — the tone of a written message (tweet, Facebook update, etc.) — and tag that text as positive, negative or neutral.

Much can be gleaned from sentiment analysis. Companies can target unhappy customers or, more importantly, find their competitors’ unhappy customers, and generate leads. I like to call these discoveries “actionable insights” — findings that can be directly implemented into PR, marketing, adverting and sales efforts.

The Limits of NLP in Social Media

As with most computer systems, NLP technology lacks human-level intelligence, at least for the foreseeable future. On a text-by-text basis, the system's conclusions may be wrong — sometimes very wrong. For instance, the tweeted phrase "You’re killing it!" may either mean “You’re doing great!” or “ You’re a terrible gardener!” No automated sentiment analysis that currently exists can handle that level of nuance.

Furthermore, certain expressions (“ima”) or abbreviations (“#ff”) fool the program, especially when people have 140 characters or less to express their opinions, or when they use slang, profanity, misspellings and neologisms.

Finally, much of social media interaction is personal, expressed between two people or among a group. Much of the language reads in first or second person (“I,” “you” or “we”). This type of communication directly contrasts with news or brand posts, which are likely written with a more detached, omniscient tone. Furthermore, each of the above social media participants likely varies their language when they choose to post to Twitter vs. Facebook vs. Tumblr. The English language and intonation differs hugely based on the source and the forum.


NLP is a tool that can help move your business forward by providing insight into the minds of your target audience members. However, it is not meant to replace human intuition. In social media environments, NLP helps cut through noise and vast amounts of data to help brands understand audience perception, and therefore, to determine the most strategic response.

Image courtesy of Flickr, weeta