SQL Server and Hadoop - unlikely bedfellows but combine powerfully

Advice Mark Wilson Nov 7, 2011

Big Data is hard to avoid – what does Microsoft’s embrace of Hadoop mean for IT Managers?

Because most enterprises have both the structured and unstructured data, we really need tools that allow us to analyse and manage data in multiple environments – ideally without having to go back and forth. That’s why there are  so many vendors jumping on the big data bandwagon but it seems that a SQOOP connector is not the only work Microsoft is doing in the big data space:

Commoditisation
In our increasingly cloudy world, infrastructure and platforms are rapidly becoming commoditised. We need to focus on software that allows us to derive value from data to gain some business value. Consider that Microsoft is only one vendor, then think about what Oracle, IBM, Fujitsu and others are doing. If you weren’t convinced before, maybe HP’s Autonomy purchase is starting to make sense now?

Looking specifically at Microsoft’s developments in the big data world, it therefore makes sense to see the company get closer to Hadoop. The world has spoken and the de facto solution for analysing large data sets seems to be HDFS/MapReduce/Hive (or similar).

Maybe Hadoop’s success comes down to HDFS and MapReduce being based on work from Google whilst Hive and Pig are supported by Facebook and Yahoo respectively (i.e. they are all from established Internet businesses).  But, by embracing Hadoop (together with porting its tools to competitive platforms), Microsoft is better placed to support the entire enterprise with both their structured and unstructured needs.

 

Mark Wilson is a Strategy Manager for a major systems integrator and an independent technology writer. With almost two decades experience of the IT industry, Mark has a background in leading large IT infrastructure projects in the UK, mainland Europe and Australia and now focuses on providing thought leadership to help customers to shape business and technology strategy.

Follow Mark on Twitter @markwilsonit .