Thinking about big data? Then think about cloud computing

Advice Jinesh Varia Jan 16, 2012

Major organisations with high data demands are increasingly turning to cloud computing to crunch the numbers.

In today’s competitive business environment, data is an asset that can be critical to success. Data can drive insights into customer behaviour, help improve quality and cost of operations, drive innovative product features and, ultimately, increase the bottom line.

For example, Climate Corporation analyses weather measurements from 2.5 million locations and forecasts from major climate models on a daily basis. It processes that data along with 150 billion soil observations to generate 10 trillion weather simulation data points. The result of these analytics is a full-season weather insurance programme that protects the $3 trillion global agriculture industry from the financial impact of adverse weather.

As another example, Yelp, the popular consumer review website and mobile application, has to store 100GB of logs per day. On top of that, the company is constantly analysing data to optimise website features such as “people who viewed this also viewed,” so customers always have a personalised experience when they visit Yelp.

These are just two examples of companies making data a core business asset and every day more businesses are discovering the value hidden in their data by leveraging cloud computing.

Big data & cloud computing

With every click, swipe, pinch, tap, like, tweet, check-in, share and API call, we are generating data. Big data is all about storing, processing, analysing, organising, sharing, distributing and visualising these massive amounts of data so that companies can distill knowledge from it, gain valuable business insights from that knowledge, and make better business decisions, all as quickly as possible.

Cloud computing ensures that our ability to analyse large amounts of data, and to extract business intelligence, is not limited by capacity or computing power. The cloud gives us access to virtually limitless capacity, on-demand, and businesses pay only for the resources they consume. In doing so, it lowers total cost, maximises revenue and gets data processing done faster at scale.

Elasticity, the ability to grow or shrink technology infrastructure resources on demand, is a fundamental property of cloud computing that drives cost benefits. While traditional data warehouses, which are tuned to answer regularly-asked questions such as generating the nightly sales report, have capacity that can be easy to predict, the analytics to discover new trends and correlations in data is an activity that requires an unpredictable amount of compute cycles and storage.

For example, to process Big Data in a traditional on-premise set up, businesses have to provision for the maximum power they might need at some point in the future. To process Big Data in the cloud, businesses can expand and contract their infrastructure resources depending on how much they need at the present moment. They no longer have to wait for weeks or months to procure, acquire and setup physical servers and storage.  With cloud computing, businesses can roll out hundreds or thousands of servers in hours.