Big data: From buzzword to crystal ball

When Song Gil-young asks visitors to his office if they would like a drink, it isn’t just a perfunctory, run-of-the-mill question. He is data mining, a special skill set that has seen him rise to the ranks of executive vice president of Daumsoft.

The company specializes in understanding human behavior and society, and coming up with predictive insights by translating social data that mostly comprises what people say on the Web - from Twitter to Facebook, blogs and myriads of other communities.

Data mining refers to the collection, storage, analysis and sharing of voluminous amounts of data collected around the world, both online and off. This is defined as big data, one of the latest buzzwords in Korea’s IT industry.

The firm’s data mining service runs on the back of its self-developed text analysis mechanism and serves a variety of purposes. Chief among these, perhaps, is the ability to predict the future, such as who was most likely to win the recent Seoul mayoral by-election.

Daumsoft correctly predicted that Park Won-soon would claim the honor after analyzing million of tweeted comments in the run-up preceding the Oct. 26 election.

The company, which once belonged to the nation’s No. 2 portal operator Daum Communications, also produces daily reports for such clients as Samsung Electronics, keeping the tech titan up to scratch on which products are being “talked” about what words commonly pop up in association with them. It also provides marketing advice for cosmetics, apparel and food companies.

“In the past, marketing relied, to a large extent, on one’s gut feeling, but now it is associated with the intellectual capability to comprehend data .?.?. We think that putting large amounts of data together under a single framework for analysis is overly mechanical, because the creators of social data are human beings,” he said.

“This is an entirely new industry, and conferences and seminars related to this have been exploding since late last year. It will bring a remarkable change to society and the corporate world.”

Here are some excerpts from Song’s recent interview with the Korea JoongAng Daily.

Q. What qualifications do data scientists require?

A. Statistical and mathematical skills are not enough because system data is only half of the job. It is their role to translate that information into insights. That’s why the capability to interpret information and see patterns in it is essential.

The Data Science Venn Diagram, which was created by [U.S. intelligence analyst] Drew Conway, defines a data scientist as someone with substantive experience, hacking skills and knowledge of statistics. Some 30 percent of my employees have degrees in psychology, philosophy, economics or religious studies. Some even have Ph Ds in linguistics.

[Demand for data scientists is soaring. According to McKinsey & Company, the United States alone will face a shortage of 140,000 to 190,000 people with deep analytical skills, as well as lacking 1.5 million managers and analysts able to analyze big data and make decisions based on their findings, by 2018.]

How has the coverage of analysis evolved over the last decade?

From around 2003, the data showed notable growth. We mostly analyzed Web forums, communities such as SK Communication’s Cyworld and shopping malls. From 2007, social media networks began popping up like mushrooms, and now Twitter is dominant. There are limits to how far you can analyze Facebook as a large portion of its users protect their privacy.

[IBM said last year that the world’s Internet users were producing 2.5 quintillion bytes of data per day.]

What are the problems with corporate marketing activities through Twitter?

What’s happening nowadays is that many corporate Twitter accounts are run by individuals or agencies hired by the company. This implies that a company is being promoted by people who don’t really have much idea about the company.

Even large firms are reluctant to set aside hefty expenses for marketing through social networking service such as Twitter. So they forge a contract with a social networking marketing agency, and then the agency hires individuals or smaller agencies to do the job. The performance is gauged by the number of tweets per day and the number of followers.

Therein lies the problem. The company doesn’t seem to care about the quality of the marketing activities, only the quantity.

How would you compare your client Samsung Electronics and, say, Apple?

The core difference lies in whether they pay attention to the human user or the device being used. To take one example, when you plug your earphones into an iPhone while listening to music with an internal speaker, the volume is automatically lowered. But when you do the same with Samsung’s Galaxy smartphone, the volume doesn’t change. This means that if you have the volume at a high setting, you are going to be in a shock.

It’s hard to say who is right and who is wrong. But Apple has tried to consider the user, while Samsung thinks it is the user’s job to adjust the volume before plugging the earphone into the jack.

* The seventh edition of Daumsoft’s Opinion Mining Workshop, which will run under the theme “From Big Data to Micro Trend,” is scheduled for May 25 at L Tower in Yangjae, southern Seoul. Interested participants can register online at www.omw.or.kr. English interpretation will depend on the number of international attendants, according to the company.

By Seo Ji-eun [spring@joongang.co.kr]