The rise of the machines immediately paints an apocalyptic image in the mind – mainly thanks to the Terminator franchise – but today our world is dominated by data-driven machines. Our PC’s, laptops, and mobile devices. Our TV’s, cars and the Internet of Things.
But machines are even more prevalent when it comes to data, especially big data. We – and of course the machines – are generating so much data that powerful computing machines are required to process and store it – and entire industries are devoted to making sense of it.
What is quite scary – depending on your point of view – is that machines are learning. Through computer algorithms they automatically upgrade themselves by discovering patterns in existing data without being explicitly programmed.
This whole processing of machine learning, rather than being sinister future vision, is now a reality and a requirement in dealing with the mountains of data we are producing. Essentially, many of our systems in business and the wider society wouldn’t work without data, machines to process the data and of course, humans (data analysts) to make sense of it all.
5 Facts About Big Data
To give a sense of how mind-boggling the numbers are when we talk about data and the growth of the big data phenomenon, here are five facts that may make your brain hurt!
- What is 2,500,000,000,000,000,000? Well, it is two and a half Quintillion. Never heard of it? Most haven’t, but this is what users of the internet generate in bytes of data each day, on average.
- What platform allows the sending of 65 billion messages per day? It’s WhatsApp. They have 2 billion users and over 5 million companies using the platform.
- The global finance sector has increased spending on big data infrastructure by $18 billion in five years. What is it worth now? Over $30 billion!
- How long would it take a person to download all the data from the internet? 181 MILLION years. Now if you turned this into miles it would take you 15 minutes to cover that distance if you travelled at the speed of light.
- What percentage of films are watched by users based on Netflix’s machine learning algorithms? Approximately 75% of Netflix users select films recommended to them by the company’s machines.
Data is Sexy
In 2012 the Harvard Business Review hailed data science as ‘the sexiest job of the 21st century’ and this seemed like an exaggerated view of those working in the world of data. Fast forward to today and every business wants to employ data scientists. The value of data, or more accurately, the value of the insights it offers have now been realised by many. The field of data science covers many disciplines, data analysis, informatics, AI, Machine learning, numerical analysis, words analysis, business analytics, and plotting.
More than anything, what data scientists do is make discoveries from data. They are comfortable in the digital economy and able to bring structure to large quantities of formless data. In a competitive landscape where data never stops flowing, data professionals help decision makers shift their focus to the ongoing customer signals that come from data.
Forbes reported that 95% of businesses cite the need to manage unstructured data as a problem for their business. The CIO highlighted that 80-90% of the data we generate today is unstructured.
The challenge has been accelerated with faster mobile networks leading to increased usage – IBM reports that 90% of all data has been created in the last two years.
This all supports the argument that skilled data professionals and more computing machine power will be required as we move into the 2020s. It is predicted that 97.2% of organizations are investing in big data, and AI and job listings for data science and analytics will reach around 2.7 million by 2020, according to recent research by Forbes.
Insights From Data
Data science extracts knowledge and insights from data, often information that would not be available from just one set of data. The term “big data” refers to data that is so large, fast or complex that it’s difficult or impossible to process using traditional methods.
The roots of data science are in scientific methods and algorithms and these are rigorously used. The output has to be verified as plausible. Key aspects are based on the ability to evaluate extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions.
Companies are competing in the data capture race. According to Globe Newswire, at the end of 2019, worldwide spending on Big Data was already worth $180 billion, and it is projected to grow at a CAGR of 13.2% between 2020 and 2022. Reports have it that IT purchases, hardware purchases, and business services could receive the highest spending on big data analytics.
The aim is to better understand customers. Data gives direction on what customers need and want.
Netflix, for example, is using data to nurture customer loyalty. The company saves £$1 billion per year in customer retention and over 100 million subscribers. Collecting huge amounts of data from users is key to their retention strategy. If you are a subscriber, you are familiar with how they send you suggestions of the next movie you should watch. This is done using past search and watch data – key insights on what interests the subscriber most.
For many companies, consumer data offers a way to better understand customer behaviour and to improve the customer experience. Data such as reviews and feedback, are used to nimbly modify their digital presence, goods, or services to better suit the current marketplace. Just think Amazon – retail, cloud computing, advertising and a continuous development of products to capture different markets.
Contextualized data can help companies understand how consumers are engaging with and responding to their marketing campaigns, and adjust accordingly. This highly predictive use gives businesses an idea of what consumers want based on what they have already done. Mapping users’ journeys and personalizing their journey, segmenting data effectively means companies market to the people more likely to engage. This helps to transform the data into cash flow. For businesses that capture large amounts of data, collecting information and then selling it, represents opportunities for new revenue streams. Data brokers, or data service providers that buy and sell information on customers. This subset of the data boom has risen alongside the big data phenomenon. Some businesses even use consumer data as a means of securing more sensitive information. For example, banking institutions sometimes use voice recognition data to authorize a user to access their financial information or protect them for fraudulent attempts to steal their information.
The Future of Data
We love generating data in the value exchange with corporations to use their services, and corporations love using data to help them make decisions. In their 2020 blog “The future of big data: 5 predictions from the experts’ 2020-2025”, Itransition offer some interesting predictions:
- Data volumes will continue to increase and migrate to the cloud – Most big data experts agree that the amount of generated data will be growing exponentially in the future. In its Data Age 2025 report for Seagate, IDC forecasts the global datasphere will reach 175 zettabytes by 2025.
- Machine learning will continue to change the landscape – Experts believe that computers’ ability to learn from data will improve considerably due to more advanced unsupervised algorithms, deeper personalization, and cognitive services. As a result, there will be machines that are more intelligent and capable of reading emotions, drive cars, explore the space, and treat patients. Investment is huge.
- Data scientists and CDOs will be in high demand – The positions of Data Scientists and Chief Data Officers (CDOs) are relatively new, but the need for these specialists on the labour market is already high. In 2019, KPMG surveyed 3,600 CIOs and technology executives from 108 countries and found out that 67% of them struggled with skill shortages (which were all-time high since 2008), with the top three scarcest skills being big data/analytics, security, and AI.
Being frightening and fascinating at the same time, the future of big data analytics promises to change the way businesses operate in finance, healthcare, manufacturing, and other industries. The overwhelming size of big data may create additional challenges in the future, including data privacy and security risks, shortage of data professionals, and difficulties in data storage and processing.
However, most agree that big data will mean big value. It will give rise to new job categories and even entire departments responsible for data management in large organisations. This means that if an organisation is to benefit from data, they need to find the right skilled data specialists, and quickly. As UK firms indicate their strongest hiring intentions in a year – a data focused talent drain is coming.
To find out how your organisation can use apprenticeship levy funding to recruit or upskill Data Technicians and Analysts, watch our free webinar: Data Heaven or Data Hell? Data Analytics in 2021 and Beyond.