Big Data and Analytics is the new 21st century gold rush and will impact every single business entity. In the time to come, the Knowledge and the Actionable Insight generated from Big Data and Analytics will overcome a sense of disenchantment and give business a competitive advantage. Business houses can leverage data in new ways, understand its hidden potential to further fine tune their strategies.
But the question is – What is this Big Data all about?
Big Data is often defined by 3Vs – Volume, Velocity and Variety.
In simple words, Big Data is an incredible amount of data (usually Terabytes, Petabytes or more), coming at an alarming or extremely high speed and which can be in structured (like data in your excel sheets defined by rows and columns) or semi-structured / unstructured form (like your Facebook posts and Twitter tweets).
Stats show that 2.5 quintillion bytes of digital data is being generated every day, data doubling itself every 1.2 years.
Big Data made waves in the analytics space because existing as well as traditional tools, techniques and infrastructure are incapable of handling this large volume and complexity of data. Hence, the term “Big Data” has expanded not only to include data but the set of technologies to capture, store, manage and analyze large and varied collection of data to solve complex problems.
Apart from this, with Internet of Things (IoT) gaining momentum, Big Data hype has grown even bigger. According to Cisco, IoT will be generating 400 zettabytes of data by 2018. In spite of all techniques and tools in place, capturing and analyzing this much volume of data is going to be a huge challenge. Presently we are able to analyse only 0.5% of the entire data generated.
At the same time, it is very easy to get overwhelmed by the term “Big” in Big Data, however the need of the hour is “Smart” Data. Once unlocked, the structured data is highly beneficial to take strategic decisions.
Big Data Management deals with 3 major challenges:
1. Strategy: Looking at new ways to use information to drive growth
2. Data Analytics: Drawing insights and trends and using this for decision making
3. Management: Controlling 3 Vs adopting process efficiencies
Big Data Opportunity in India
Burgeoning size of digital information triggered by social media, ecommerce, banking and 900 million mobile connections out of which 100 million are data users is resulting in 83% of annual growth rate of Big Data in India.
Big data has found vertical market applications, ranging from fraud detection/risk management to scientific R&D, along with proliferation of data from multiple real time sources.
Horizontal Submarkets for Big Data
– Storage & Compute Infrastructure
– Networking Infrastructure
– Hadoop & Infrastructure Software
– Analytic Platforms & Applications
– Cloud Platforms
– Professional Services
Vertical Submarkets for Big Data
– Automotive, Aerospace & Transportation
– Banking & Securities
– Defense & Intelligence
– Healthcare & Pharmaceutical
– Smart Cities & Intelligent Buildings
– Manufacturing & Natural Resources
– Web, Media & Entertainment
– Public Safety & Homeland Security
– Public Services
– Retail & Hospitality
– Utilities & Energy
– Wholesale Trade
Hurdles and Prospects of Big Data and Analytics projects
While all the hypes are true and need is indeed there, however 99% (Yes you read it right!) of Big Data and Analytics implementation fail. The top 5 reasons for same are:
1. Failure to define use case in objective terms
2. Failure to use right technology
3. Failure to focus on business requirements first, technology next
4. Failure to leverage all available data sets and assets
5. Failure to effectively use power of advanced analytics
Hence, not only technology but the people are the key drivers for Big Data and Analytics space. Most of the organizations are making investments on building human capital, people who know tools, business problems and techniques to solve complex problems. This has led to birth of new profile “Data Scientist” – which is one of the hottest skills to have in industry as of now.
Keep watching this space for more on Big Data, Analytics, Data Scientists, Machine Learing and much more.