We all know that Big Data is getting a lot of attention these days. Rather than focus on the technology today, I was curious about what Gartner had to say about the future of Big Data. More specifically what Gartner has to say about Big Data Spending for 2013 and beyond. Gartner released a report in October. I am glad to share some highlights with you.
First letâ€™s put this whole Big Data thing into context. Data is exploding at an astounding rate. Â I use the following diagrams often when speaking with my customers to describe essentially the evolution of Big Data. I find these simple enough that even my 11 year old gets understands Big Data. Believe it or not, some of my customers are scratching their head when looking at these pics, but thatâ€™s a story for another day. While it took from the dawn of civilization to 2003 to create 5 exabytes of information, we now create that same volume in just two days! By 2012, the digital universe of data will grow to 2.72 zettabytes (ZB) and will double every two years to reach 8 ZB by 2015. Here are 3 key facts you need to know about Big Data.
Big data refers to huge data sets characterized by larger volumes (by orders of magnitude) and greater variety and complexity, generated at a higher velocity than your organization has faced before. These three key characteristics are sometimes described as the three Vs of big data.
Big data analytics is a technology-enabled strategy for gaining richer, deeper, and more accurate insights into customers, partners, and the businessâ€”and ultimately gaining competitive advantage. By processing a steady stream of real-time data, organizations can make time-sensitive decisions faster than ever before, monitor emerging trends, course-correct rapidly, and jump on new business opportunities.
Big data spending will drive $28 billion of worldwide IT spend in 2012,Â according to Gartner, Inc. In 2013, big data is forecast to drive $34 billion of IT spending.Â Most of the current spending is used in adapting traditional solutions to the Big Data demands â€” machine data, social data, widely varied data, unpredictable velocity, and so on â€” and only $4.3 billion in software sales will be driven directly by demands for new big data functionality in 2012.
Big data currently has the most significant impact in social network analysis and content analytics with 45 percent of new spending each year. In traditional IT supplier markets, application infrastructure and middleware is most affected (10 percent of new spending each year is influenced by big data in some way) when compared with storage software, database management system, data integration/quality, business intelligence or supply chain management (SCM)Sales and marketing campaigns will rely heavily on the ability to extract business intelligence, customer trends and analyze what their target market is looking for in a product or solution. As the world becomes more dynamic, the dependency on Big Data will become bigger than ever. By 2020, Big Data features and functionality will be non-differentiating and routinely expected from traditional enterprise vendors and part of their product offerings.”
Research vice president at Gartner Mark Beyer has stated that “In 2011, big data formed a new driver in almost every category of IT spending. However, through 2018, Big Data requirements will gradually evolve from differentiation to ‘table stakes’ in information management practices and technology. By 2020, Big Data features and functionality will be non-differentiating and routinely expected from traditional enterprise vendors and part of their product offerings.”Â “As a result, Big Data will once again become ‘just data’ by 2020 and architectural approaches, infrastructure and hardware/software that does not adapt to this ‘new normal’ will be retired. Organizations resisting this change will suffer severe economic impacts.”
My Point of View
Big Data should not be treated as a service for which is being provided by IT. In my point of view Big Data is the ability or initiative to extract intelligent information required to effectively run your business and provide a competitive advantage. I find it difficult to imagine how a non agile IT Department or cost center (IT as we know it) can provide this. This is why I believe the business will become more involved in the IT decision making or perhaps take it one step further where the CMO owns these types of initiatives.
Starting near the end of 2015, Gartner expects leading organizations to begin to use their big data experience in an almost embedded form in their architectures and practices. Beginning in 2018, big data solutions will be offering increasingly less of a distinct advantage over traditional solutions that have incorporated new features and functions to support greater agility when addressing volume, variety and velocity. However, the skills, practices and tools currently viewed as big data solutions will persist as leading organizations will have incorporated the design principles and acquired the skills necessary to address big data concerns as routine flexibility.Â 10 â€“ 15 % of organizations will take full advantage of Big Data. Those that do will outperform their competitors by 20% across major financial metrics. Thatâ€™s not big, that is HUGE!
One thing I keep hearing about Data Science are the skills required to become a Data Scientist. With the technology required to process this data, will come the spending on human resources. These resources will be the Data Scientists assigned to triage and find value in the data collected and stored. I often hear that a good mathematical, statistics and programming background are essential for Data Scientists. Although I believe these qualities are important, I believe a Data Scientist should also have the necessary experience to put that data captured and processed into the business context. Finding skilled personnel is one of the major challenges associated with big data analytics. Successful Big Data analytics initiatives involve close collaboration between IT, business users, and â€œdata scientistsâ€ to identify and implement the analytics that will solve the right business problems. Data science is an emerging field, and data scientists are a new kind of professional with a unique skill set. Data Scientists are responsible for modeling complex business problems, discovering business insights, and identifying opportunities. Demand is high for people who can help make sense of the massive streams of digital information pouring into organizations.
Additional information is available in the report “Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016”. The report is available on Gartner’s website atÂ http://www.gartner.com/resId=2195915.