In a rare interview, Google Chairman Eric Schmidt gives Bloomberg his outlook for 2014 trends (http://www.bloomberg.com/video/ask-a-billionaire-eric-schmidt-s-2014-predictions-pmV~qd7qTeipbjKx6_wW1Q.html). In this 2 minutes video, Schmidt talks about several obvious trends, like “everyone will have a smart phone… essentially connected to a super computer”. Another important remark Schmidt made is about the “arrival of big data and machine intelligence everywhere”.
It is not about the invention of big data and machine intelligence. It is about the arrival. I find that to be the essence of the big data revolution. The idea that we can leverage computers and technology to process data and derive information and intelligence from thatdata is certainly not new. However, it has been hampered by slow processors, lack of storage capabilities, and inability to amass sufficiently significant amounts of data to deliver this intelligence. Past attempts to overcome the technology shortfalls have been painful and could only deliver meaningful results when applying large investments. Well, all that is changing now. It is now affordable and relatively simple to perform analytical tasks that were considered extremely challenging only a few years ago. My news analytics experiment is one example of that (http://bihappyblog.com/2013/12/05/predicting-the-news-with-sap-hana/). IT departments and organizations are finding that barriers that prevented them from delivering google like level of services are quickly disappearing, and the bar set by consumer web sites for level of service, sophistication and relevance can now be met with of enterprise applications. This is the challenge we in the BI industry will have to face and meet over the next few years. I believe that leveraging the knowledge and understanding developed during decades of data warehousing development to overcome technological impossibilities positions us, the BI industry veterans in a unique position to help drive the big data revolution in the corporate world. Modeling, abstraction skills, problem solving and technical know-how are the building blocks needed for any big data related project, and these are the traits of the trade for us in the BI industry.