Data scientists have earned their seat at the table
When I started working at Quantium in 2005, data science and AI were niche capabilities exploited by a relatively small cross-section of businesses. It was a time of “bespoke analytics” when each analytics project was painstakingly developed from scratch. The potential of data to drive better products, better services and better experiences was understood by some, but its realisation was limited.
Just over a decade on, it is a very different world. In the same way that it is no longer possible to find a company of significant size without a CFO and a finance division made up of specialist accountants, so too are companies starting to see data science and data-driven decision making as a critical part of doing business.
The result is a surge in the need for data science capability, and we have only begun to scratch the surface of the possibilities available.
Let’s take an analogous advancement. When electricity was first rolled out to private households, the main application was the use of electric lighting to replace gas lights. Gas lights had the obvious down-sides of needing to be lit by hand and of causing the occasional house fire.
Electric lights were patently superior – but who could have foreseen the other uses of electricity in the home? TVs, washing machines, even electric toothbrushes? These days, lighting makes up less than 10% of domestic electricity usage. So too will a vast array of unexpected data science and AI applications surprise us.
One factor that previously limited the realisable potential of data science was the amount and quality of data available.
As technology advanced and every industry adopted the computer as a fundamental tool of commerce, and as consumers began using mobile devices to assist with every aspect of their lives, the problem of generating data was suddenly solved. The quality of that data has also improved continually as businesses come to understand how to collect and categorise data in a form better suited to giving them the insights they need.
As businesses embed this enhanced understanding into their value chains, they are becoming more responsive to the needs of their customers across all functions. The intelligent application of data means more personalised customer communications and recommendations, more targeted marketing, more intelligent pricing, more accurate demand forecasting and reduced fraud.
Some services wouldn’t be possible without the data science-enabled business model that powers them. Uber only exists because they can predict demand in advance and guide cars to those areas ahead of time where there is greater need. And other services are by their very nature powered by data science – such as web searching and the personal assistants now available in every smartphone.
The services and business models that data science has made possible have spawned nimble newcomers to almost every industry who are taking on the traditional market leaders: they are the disruptors, the businesses that are using technology, data and data science to beat established players at their own game.
No business is immune from the impact of disruption, which is why every business must become adept at disrupting the status quo. Even companies with a track record of innovation need to adopt a new perspective to doing business in the disruption economy.
Disruption is different to innovation. It isn’t about doing what you do better; it’s about a completely different way to solve the same need. Who needs paper street directories when you have digital maps with GPS on your phone? If a company is too narrowly focused on what it sells, and oblivious to the basic need its product addresses, it is vulnerable to a new entrant with a new and better way to address that same need.
Being a disruptor does not mean you have arrived at your destination. The disruption journey never ends. There are examples all around us of businesses or digital entrepreneurs that successfully harnessed a disruptive technology to win market share, only to be themselves disrupted and fade into insignificance a few short years on.
The dynamism of disruption means data science and AI can no longer be treated as one-off investments or as an esoteric component of the business hidden in the basement. As data science cements its position as a cornerstone of business in the 21st century, data scientists have earned their place at the decision-making table.
In the scheme of things data science and AI are still nascent. We have broken away from our prior beliefs about what is possible, but we are still discovering new ideas, new paradigms, and new frontiers of possibility.
It’s a great time to be a data scientist.