IDC Study on Hadoop Confirms Big Data Customers Use More Than Hadoop to Holistically Analyze Enterprise Data

Hadoop is gaining incredible traction in enterprise big data implementations1. But how do enterprises use Hadoop in concert with traditional analytics to gain actionable information from their big data? What impact does this goal have on Hadoop deployment decisions?

An IDC study commissioned by Red Hat, titled “Trends in Enterprise Hadoop Deployments” reports, “... 32% of respondents indicated that their firms have existing Hadoop deployments. An additional 31% indicated that they had plans to deploy it within 12 months. And finally 36% said that their Hadoop deployment schedule could go beyond 12 months.”

Enterprises are using more than Hadoop for big data analysis. The IDC study states, “Nearly 39% respondents indicated that they use NoSQL databases like HBase, Cassandra and MongoDB etc. while nearly 36% indicated that they use MPP databases like Greenplum and Vertica etc. This situation also underscores the importance of causality and correlation – in which traditional structured data sets are analyzed in conjunction with unstructured data from newer sources.”

The IDC study also explores the use cases for enterprise Hadoop deployments and confirms that “... businesses use Hadoop in more than one way:

• Analysis of raw data – whether it is operations data, data from machines or devices, point of sale systems or customer behavioral data gathered from ecommerce or retail systems – is one of the dominant use cases for Hadoop.
• Nearly 39% of respondents indicated that they use Hadoop for service innovation, which includes the analysis of secondary data sets for modeling of “if-then scenarios for products and services.
• Some of the less popular use cases for Hadoop include its deployment as a platform for non-analytic workloads (for example in conjunction with a SQL overlay for OLTP).”

As a result, enterprises are looking to alternative persistent storage systems. According to the IDC report, “File systems like IBM’s Global File system (GPFS), Red Hat Storage (GlusterFS), EMC Isilon OneFS and others that have earned a reputation for their robust, scale-out capabilities are clearly preferred as alternatives to HDFS. Of these three, only Red Hat offers an integrated open-source based enterprise Linux platform that combines a distributed file system with a Hadoop connector, enterprise middleware and the ability to run Hadoop computational workloads natively.”

The survey also found that most enterprises process big data both before and after Hadoop processing. This highlights another attractive feature of GlusterFS and other alternatives – the ability to keep the data in native POSIX format and use traditional analysis tools.

To download the full copy of the IDC Study, register here.

1IDC White Paper, Trends in Enterprise Hadoop Deployments, commissioned by Red Hat, August 2013