McKinsey finds hard work to do in Big Data Revisited report

In 2011, the McKinsey Global Institute published a landmark report on big data that captured the attention of C-level executives in a manner unusual for an IT topic.

The firm’s research wing revisited the report last year, publishing The age of analytics: competing in a data-driven world in December 2016.

The report found that while a few – mainly digitally native – companies are racing ahead with data and analytics, most are capturing, at most, 30% of the value they could. Sub-optimal organisational design lies at the heart of that failure, and McKinsey asserts how hard data-driven digital transformation is to do.

One of the report’s authors, Michael Chui, said: “Technology is not an area where things happen overnight. What we wanted to do was assess how far we have come over the past five years.”

And of big data analytics’ relative failure to make a big impact in most companies surveyed in the firm’s research, he said: “Many times there was success, and really useful, value-creating insights [in analytics programmes], but the issue was how to turn those insights into changes in how the organisation operates so that it turns up in the bottom line.

“It is hard work, this, but there is a parallel with lean management. It has been done before, this scale of change.

“Even the digital natives are trying to get to the next level. And [more established] incumbents are all at various stages of a journey as to how to use data to be more competitive. Some have invested more in digital talent than others.

“The markers of transformation are when companies move from changing what they are selling – for example, from maintenance services to guarantees of uptime; or in healthcare, a shift from medical procedures to keeping a population healthy.”

The report found companies are capturing only a small amount of the potential value from data and analytics. The 2011 report estimated this potential in five domains: location-based data, US retail, manufacturing, the EU public sector and US healthcare. The institute found progress in location-based services and in retail, both areas with digital native competitors. In contrast, manufacturing, the public sector and healthcare have captured less than 30% of the value the firm highlighted five years ago.

Relatively new, from 2011, is a focus on artificial intelligence, in particular that type of AI that is machine learning, where computers have the ability to learn without being programmed. “There is a lot excitement about those technologies,” said Chui.

The report’s authors suggest recent advances in machine learning can be used to solve a variety of problems, and that deep learning, by which machines emulate how humans learn, is pushing the boundaries even further. “Systems enabled by machine learning can provide customer service, manage logistics, analyse medical records or even write news stories,” they say.

But the report seems to highlight a lack of progress in companies doing what might be called “hard digital”, involving complex data integration or the digitising of supply chains – as opposed to, say, the location-based services offered by retailers to smartphone users.

The report found, for instance, senior management in manufacturing to be sceptical of the impact of data analytics programmes, and their companies to be hamstrung by siloed data in legacy IT systems. Only an estimated 10 to 20% of the value McKinsey posited, in 2011, that could be derived from data analytics in lower product development and operating costs in manufacturing is now being secured, five years on.

“There is a lot of hard work to be done,” said Chui. “We found those who had moved furthest were those who had digitally native competitors pushing them on.” More challenging are areas such as machine predictive maintenance – important in oil and gas, among other industrial sectors, he said.

“What happens when the entire repair organisation is paid on the basis of being heroes who fix things?” he said. “How do you change that so that they are still heroes when nothing breaks?”

Chui is based in San Francisco, and Silicon Valley is the heartland of much big data technology. What has been the impact of the report there? “The dialogue is in capturing more of the value from data,” he said. “There is a sense that there is too much data wrangling [to get data into shape for analysis] just now. The main thing is our perspective this time around was on how far we have come, and what the obstacles have been. The analysis on the need for [data analysis and data science] is still germane.”

The report found that attracting and retaining the right talent – not only data scientists but also “business translators” who combine data savvy with industry and functional expertise – to be an enduring problem.