Is Sentiment Analysis Based Solely on Algorithms Truly Valuable? 【Dr. Cheong's Data Insights 10】

Limitations of Automated Sentiment Analysis

In the tech world, artificial intelligence (AI) and algorithms have become popular keywords in recent years. Many social network platforms and sentiment analysis service platforms claim that using AI and machine learning algorithms can accurately push information and determine the positive or negative sentiments of netizens. However, it is well known that fake news and fake ads continue to flood these platforms and cannot be effectively curbed, sometimes leading to very embarrassing situations.

Recently, after the Las Vegas shooting, a user on the forum 4chan falsely claimed that the shooter was another man named Geary Danley (later quickly confirmed to be Stephen Paddock). The false information spread across the internet, and Google placed this news at the top of search results, becoming an accomplice in spreading fake news. Similarly, incorrect information also appeared on Facebook's crisis response and safety check pages. On another social platform, Twitter, incorrect comments about the case were not spared either.

After the incident, Google, Facebook, and Twitter all claimed they would improve their algorithms and increase manpower to control misinformation. Facebook announced a few months ago that it would recruit 7,500 content moderators to help filter out fake news. Recently, it also claimed to add 1,000 content moderators to monitor automated ads, suspecting that Russia used automated accounts to spread ads during last year's US presidential election.

Why did these global information search and social network giants all make the same "fake" mistake? Did those AI and algorithms fail?

Even Google Needs Human Monitoring

In fact, from the embarrassing situations these giants encountered and the need for a large amount of human monitoring (by the way, Google News is more accurate because it is manually reviewed, unlike Google Search, which relies purely on algorithmic automation), it illustrates a critical issue: current automated semantic judgment methods (including various algorithms and machine learning methods) largely fail to accurately identify false or incorrect information, meaning they cannot effectively interpret the meaning of textual content.

Positive and Negative Sentiments Cannot Be Interpreted as Support or Opposition

For a while, I have been asked multiple times by industry players and clients about the accuracy and value of automated analysis of online content (or text mining). When discussing these issues, the general public's understanding still stays at the level of sentiment analysis, which is the positive or negative emotions expressed by the content. It has not reached the level of judging the intensity of emotions, such as being very angry or quite fond, or the level of support or satisfaction with a certain event, brand, or person. Let alone understanding the logical relationships in different contexts, or the implied meanings in slang, idioms, and sarcastic language.

So, how is automated sentiment analysis done in the industry? Generally, it uses a combination of Natural Language Processing (NLP) and Text Mining technologies, through lexicons marked with positive polarity and negative polarity and assigned weight values. The entire or part of the content to be analyzed is classified into bipolar (positive, negative) or three categories (positive, negative, neutral) to judge the sentiment attitude expressed in the text. It is then classified as positive, negative, or neutral based on the emotional attitude that can be parsed from the text.

Social Media Netizens' Language is Varied and Difficult to Identify Reference Objects

However, this purely automated sentiment analysis has a fatal problem. For example, social public opinion from netizens mainly refers to user-generated content (UGC) from social platforms. The range of topics is broad, and the language used is irregular. Different authors' narrative styles often vary greatly, and the content itself may not be sufficient to judge its emotional attitude (sometimes it may be judged through comments from other netizens). Here, automated sentiment analysis is a fixed procedure based on specific content sources and topics. In practice, it can only analyze positive or negative attitudes, making it difficult to identify the referenced objects.

Negative Netizen Attitude: How to Respond?

Often, when analyzing a netizen's comment, it is difficult to determine whether the sentiment is directed at a particular event, person, or brand. Even if the analysis accuracy reaches over 90% (generally, 70% to 80% is already quite good), it is meaningless for insight or decision-making. If a marketer or policy decision-maker sees that netizens' emotions are negative but does not know what the negativity is directed at, how can they respond? Furthermore, a commonly overlooked aspect is that these inherently non-referential positive or negative emotions, if expressed in percentage distribution, can be misleading as support or satisfaction with the government, brand, or person, potentially leading to decision-making errors.

Therefore, whether it is algorithms or AI, under the current limitations, human intervention is essential to make sentiment analysis truly valuable. How to intervene will be discussed in the next issue.

Dr. Angus Cheong Chairman of the Asia-Pacific Internet Research Alliance and Chief Data Consultant of uMax Data

(Originally published in the Hong Kong Economic Journal, reprinted with permission)