Mohit Sharma
Mohit Sharma
9:00 pm on Aug 8, 2013


Mohit Sharma is the lead, marketing and business development function at PromptCloud. It’s is a Bangalore-based firm dealing with big data solutions such as large-scale web crawling and data extraction. The views expressed are his own and do not reflect those of his employer.

With a petabyte of data being generated every 11 seconds, close to 90 percent of all the data in the world today has been created just in the last two years. With the dawn of user-generated content (UGC), Eric Schmidt opines that every two days we create as much information as we did from the dawn of civilization up until 2003. The data sector is expanding at an unimaginable rate.

Big Data is characterized by the three Vs: volume, variety, and velocity. Volume refers to the amount of data; variety is the number of types that the data has; and velocity refers to the speed of handling and processing data. For the sake of simplicity, big data refers to a collection of chunks of complex information that require specialized software tools to be analyzed. This data is collected in real-time from sources such as RFID sensor data, transaction records, social media profiles and updates, GPS/cellular signal data, etc. World over, organizations have come to realize the importance of this data, though most of them are still perplexed as to how they can incorporate such data-derived insights into their business strategy.

Big data adoption in Asia

When it comes to adoption of ‘Big Data as a Service’ (BDaaS), Asian companies have, for a long time, been fence-sitters, especially when compared with their western counterparts. However, that is changing fast; according to recent research conducted by The Economist’s Intelligence Unit (PDF), a majority of Asian C-level executives feel that technologies such as cloud computing, big data analytics, Software as a Service (SaaS), enterprise mobility, IT security and social media will have the greatest impact on their business strategies in the coming decade.

However, there exists a mismatch between this data point and the actual adoption rate of these technologies by large companies. The research further quotes three reasons for what is holding them back: “a lack of proven success metrics; misalignment between current business processes and the outputs of new solutions; and the diversity of the technology landscape in the Asian market in which they operate.”

Big data in Asia stands at a critical juncture, as there are a few early adopters, but a gap needs to be filled for it to become mainstream. At our level, we feel that Asian executives are quickly grasping the importance of big data and have come to realize the impact that such data can have on their revenues.
With a growing middle class, leading to a rise in consumerism (and e-commerce) in Asia, the number of companies looking to collect and synthesize data regarding consumer behavior and sentiments has also gone up. For instance, we witness requests from our Asian clients to extract geo-specific data that is useful in monitoring of brands, gauging competition behavior, or analyzing consumer sentiments through various social networking sites and blogs. We have perfected our engine to incorporate such queries to extract keyword-based or geo-specific information across most of these platforms, almost in real-time.

Challenges for big data analytics in Asia

Like most upcoming technologies, big data has its own set of challenges to overcome.

  • One major challenge for big data in Asia is that big data has a bigger learning curve associated with it, particularly with respect to use and operations. A very common issue is how one can merge the new data sources with the existing ones, and quite often companies take the easy way out – ‘let’s have both’. This causes integration issues, and also adds to the complexity associated with the use of data.

  • Multi-linguality across the continent poses a significant technology barrier for clients looking to extract web data. At PromptCloud, we have developed our crawl and extraction engine to overcome this challenge – we now daily extract data across 10 to 15 Asian languages.

  • Another hurdle that exists at the operational level for the big data industry is a lack of skilled professionals, having expertise in big data and cloud computing technologies. This talent shortage is set to exacerbate if the growth in the available talent doesn’t match the growth rate of the industry.

  • DataH

    Good article Anh-Minh. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data lifecycle. Designed by data scientists, ECL is a declarative programming language used to express data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. More at

  • BIGDATAObserver

    There is indeed a lot of challenges, but one can also see this as opportunity. Entrepreneurs who are in this space will take the first steps to address the gaps. Watch out for the likes of Pulse’s BOB CHUA who has jumped on this bandwagon with his BIG DATA practice PULSATE, and some of the other players across asia starting to gain funding and traction.


Read More