Big data goes mainstream

Great article in InformationWeek on Big Data. To be honest, most of the referenced examples are just big data warehouses and not extended beyond that to streaming data and large distributed file systems, which I think will be the norm in the future, but it still provides some interesting examples of where people are today.

I’ve spent a lot of time with 3UK, referenced in the article, who went with the IBM Smart Analytic System. They are doing some very interesting stuff, beginning with their model-driven approach to data integration, and extending all the way out to their recent investments in in-database analytics. What they are doing with their warehouse investment is very cool.

What the 3UK example shows that many of the other examples in this article fail to show is that “big data” isn’t just about having a big data warehouse and running conventional reporting and BI against it. It is really about running analytic models against data that would be too big and too expensive to analyze conventionally. The analytic models allow you to find nuggets of value by sifting through huge amounts of information. Some of the examples in the article capture this thought, but most are just big data warehouses. I think you can really tell the difference in the warehouse vendors by looking at their vision and investments around analytics. Analytics is the future of the industry. The ones making the investments today will be the leaders of the next wave of change.