Using Big Data Analytics Tools for Business Decision-Making 【Dr. Cheong's Data Insights 6】

In the era of big data, especially with the advent of social media, netizens can express their views on social events or products and services at any time through text, pictures, or videos. This has resulted in a vast amount of unstructured user-generated content (UGC) in the online world, leaving various behavioral traces. These real-time, diverse, and massive amounts of netizen opinions and behavior data have become increasingly important for today's business decisions and competitive analysis. Effectively analyzing this data to gain valuable insights is a challenge many companies face.

Data Needs Refining: Management and Analysis are Key

For big data to truly realize its potential value, two important processes are necessary: Data Management and Analytics. Data Management refers to the process of collecting, storing, extracting, cleaning, labeling, and integrating raw data into analyzable data. Analytics involves a set of tools that include various techniques, methods, and strategies to mine the meaning from the data, thereby extracting insights that aid decision-making.

Although more and more big data analytics tools are emerging on the market, not every company or business department has the same needs. Some tools are designed to analyze various behaviors of netizens browsing websites, such as Google Analytics. Others integrate data from different sources and types, such as market activities, sales performance, and operational performance, and generate visual reports, like Watson Analytics. There are also tools specifically for deep mining of netizen opinions, such as the AI-assisted online opinion mining platform mentioned in my previous article.

Choosing Different Analytical Tools Based on Purpose

Currently, the most popular big data mining tools are text analytics or text mining. This is because companies themselves or through third-party service providers can relatively easily obtain netizen comments on social media, email content exchanged with users, customer suggestion boxes, online news, forums, interview content, and internal dark data—unstructured data collected for record-keeping but not for immediate practical use. Text analytics tools can also be divided into information extraction, text summarization, question answering (QA), and opinion mining based on their analysis objects and purposes.

Information extraction generally involves extracting certain fixed terms from a large amount of unstructured data to form structured data. For example, in a promotional copy, extracting the product name, company address, and phone number is usually done using keyword matching. Text summarization uses algorithms to extract what is considered important from numerous documents to form a summary or recompose new sentences based on natural language processing techniques to form a summary, usually for the purpose of providing an overview of the text. QA tools refer to question-and-answer interactions in natural language scenarios, such as Apple's Siri and Google's voice assistant, which achieve this through information extraction and semantic judgment methods.

Sentiment Analysis: Every Sentence Matters

As for opinion mining, commonly known as sentiment analysis, it has become the most popular application in recent years. Whether it's marketing personnel, financial analysts, government public opinion analysts, or academic researchers, a lot of resources have been invested in this tool. Due to technical and tool performance limitations, the industry typically only performs positive and negative analysis at the document level, making the analysis object ambiguous. For example, in a social media opinion piece, netizens may mention more than one brand or type of service. In this case, it is unclear whether the sentiment analysis results refer to product A or product B, or if they are directed at the service staff's language or appearance.

Therefore, to delve deeper into opinions, it is necessary to analyze at the sentence level or category level to fully understand the sentiment. This way, it becomes clearer how netizens evaluate individual brands or services, facilitating precise judgments and effective decisions.

Dr. Angus Cheong Chairman of the Asia-Pacific Internet Research Alliance and Chief Data Consultant of uMax Data

(Originally published in Hong Kong Economic Times, reprinted with permission)