InsightsPredictive analyticsAI and predictive analytics: What does the future hold?

AI and predictive analytics: What does the future hold?

Advances in AI and predictive analytics will have ramifications beyond business's technological capabilities. Organizations will face new challenges in terms of skills, implementation and more. How can marketers prepare for change?

Advances in AI and predictive analytics will have ramifications beyond business’s technological capabilities. Organizations will face new challenges in terms of skills, implementation and more. How can marketers prepare for change?

In this series we have seen how the evolutionary progress of the analytics industry leads naturally to the integration of artificial intelligence (AI) to create accurate predictive models.

In the first article, we explored the potential for AI and predictive analytics as marketing tools, driven by technological improvements that have moved from rule-based automation to something closer to sentience. We see examples of this everywhere, from apps like Google’s Waze, to financial fraud protection and the personalized recommendations on Amazon.

The second article of the series was grounded in concrete use cases for this technology, showcasing a wide variety of businesses that have used machine learning and AI to predict consumer behaviors and create better business outcomes. These opportunities are open to all companies now, but seizing them is a more complex task than merely purchasing some new software.

The third and final article of our series will focus on the future of predictive analytics and the challenges the industry faces.

Implementation challenges

Although it may seem inevitable that such a powerful business tool will be adopted en masse, the reality is more nuanced than that. Executives want smarter, faster decisions, but there is something of a high wire trapeze act in the balancing of data, people, and technology when it comes to transforming a business to an AI-driven predictive analytics model.

Implementing this technology requires an ideological shift for businesses, not just capital investment, and staff training from the ground up in data analytics is imperative.

This comes at a time when executive confidence in their organization’s digital expertise appears to be dropping. The most recent Digital IQ survey from PwC shows a decline in executives’ confidence in their team’s abilities:

The level of expertise is not falling away; the industry simply changes so quickly that staff are not keeping up.

Therefore, if AI-driven predictive analytics is to assume a central position in the CIO/CMO’s toolbox, a few substantial hurdles will need to be overcome.

Within this final part of our series on predictive analytics, we will outline some of the challenges facing this industry in future, before discussing the solutions businesses can start to implement today.

Data quality

In a recent survey of senior executives by Protoviti, data ranked as the greatest inhibitor of widespread adoption of predictive analytics within companies. Quality was one of the foremost modifiers used to add specificity to a term as nebulous as ‘data’.

Even ‘quality’ requires some further definition before we can decide on how to tackle such a gargantuan challenge.

High quality data will be consistent in its format (even at significant scale), reflective of the real world scenario it describes, and will enable reliable, reproducible research.

We can take as our example a data set of train departures from Waterloo from 2010-2014 that contains gaps across timeframes and uses inconsistent naming conventions. Humans struggle with gaps in data sets like this, but we can adapt and perhaps even procure the data from elsewhere. Artificial intelligence simply can’t work with incomplete data like this, as it can only work with what is fed into the system.

The best AI technologies in the world can only make use of the data we provide, so it is crucial that businesses are aware of these potential pitfalls and know how to avoid them. More data typically means better results from AI-driven predictive analytics, but it needs to be the right data to answer the business problem you aim to solve.

Having the right team in place is a great way to start on that path.

Recruiting and training for the right skills

Predictive analytics technology is growing in sophistication, but the level of knowledge within the industry is not necessarily advancing at the same pace.

A Capgemini report found that 77% of companies see the lack of the right skills as the biggest hurdle to successful digital transformation:

A ClickZ report went deeper to identify the skills areas that were most desirable, compared to their current level of sophistication within organizations.

It was no surprise to see analytics ranked as the most important skill, given its potential for use in every marketing function. It was perhaps a little more surprising to see analytics as the area with the biggest skills gap.

Source: ClickZ

Analytics encompasses a variety of techniques and types of data investigation. Most analytical work undertaken today falls under the banner of either descriptive (what happened?) or exploratory (why did it happen?).

Although the skill level needed to operate the technology behind future predictive analytics systems will likely lower over time, businesses still need to ensure their staff have detailed knowledge of data analytics before investing in some new, exciting artificial intelligence systems.

Luckily, there are plentiful resources and qualifications to aid with this training, as long as businesses are willing to invest. Both theory and practice should be considered fundamental components of this training.

In Analytics: How to Win with Intelligencethe authors posit that an analytics Center of Excellence should be formed in larger companies, with the department head reporting to the CTO:

The aim of this approach is to give analytics a clearly defined base from which its experts can can teach others within the organization.

We can look at this from another perspective, however. Not everyone on a marketing team needs to know the internal workings of an analytics platform in order to benefit from it. This becomes increasingly true as these platforms become dependent on machine learning to create predictive models.

Regardless: a broad knowledge base is still essentialWithout having the ability to ask the right questions or to know what the technology is capable of (and what it isn’t capable of), the outputs will not be fit for purpose.

There is therefore a growing school of thought that liberal arts backgrounds will be an increasingly important complement to statisticians and engineers. The capacity to pose the right questions as the frame for a hypothesis and then to investigate the findings will be essential, as will the softer skills required to present them to senior stakeholders.

In essence, it takes a village to get analytics right nowadays. But ensuring the quality of your data is fit for purpose and that you have a balance of skillsets in your analytics team is a great start.

Data management

There is no shortage of data in the modern age, and the quantities will only increase as Internet of Things (IoT) devices continue to make their way into homes across the world.

Every company has a potentially lucrative source of proprietary and third-party data at their fingertips. Cloud-based solutions, which can store data remotely in huge quantities, go some way to answering the question of where data should be kept.

However, even if a business uses a data warehouse like Hadoop, the information still needs to be transferred to an analytics platform and transformed into insights via statistical models.

The question of how exactly to ensure that analytics platforms and AI systems keep up remains a puzzle for many businesses.

There are other challenges with data management, too – from data mining to storage and ultimately to transforming the information into useful insight.  

2013 paper by scientists from George Washington University and American University, entitled Big Data: Issues and Challenges Moving Forward, summarized these potential issues:

With the upcoming launch of the EU’s GDPR regulation, these questions are more important than ever before. It is a business’s responsibility to ensure that all data is compliant with local laws and to dispose safely of data that does not comply.

If one thing is for certain, we cannot leave it to AI to make these calls. AI predictive models will assess whatever historical data is presented to them and, should a company notice later that erroneous data was fed into their AI analytics platform, any conclusions it reached will have to be ruled invalid.

Retracing the steps of such complex calculations and debugging any unwanted variables would prove an impossible task. As a result, any businesses planning to feed big data into an AI-based predictive model should be cautious with their data sources.

Image: Wikimedia


This category serves as an umbrella term for a range of minor – but still important – challenges.

AI and predictive analytics have clearly defined and important roles in industries such as healthcare. 80% of hospital leaders view this field as “important”, and it’s easy to see why. Any tool that can spot historical patterns related to diseases and predict their future behavior will prove invaluable in this field.

The case is not quite so clear-cut when applied to marketing. Undoubtedly, we could all benefit from accurate predictions based on the past. That applies to all of society, in fact.

However, there is an argument that predictive modeling has some limitations in an industry that thrives on new ideas. The temptation with sophisticated AI systems and predictive models is to cede control and stick to what we know will continue to deliver growth.

Additionally, predictive analytics can become a self-fulfilling prophecy. We see that a certain message, product, or audience segment is projected to convert at a higher rate, so we shift budget to capitalize on this. If the prediction comes to pass, is that because the prediction was accurate, or because we acted to make it accurate?

Finally, we should consider the role of human creativity in this process. As we discussed in our article on AI-driven content creation, the human capacity to innovate and devise new, creative solutions is one that AI cannot quite master yet. As such, we need to use technology to free up our teams to make the most of their ability to strategize for the long term.

As with any AI technology, one of the most crucial factors for success is the role that people will play to get the most out of the tools at their disposal. Looking specifically at predictive analytics, this means ensuring the right balance between quality data, the best technology, and people with the ability to know the technology’s limitations.

This concludes our three-part series on AI and predictive analytics. If you missed the previous two installments, follow the links below for a recap:


US Mobile Streaming Behavior

Whitepaper | Mobile US Mobile Streaming Behavior


US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups

Whitepaper | Analyzing Customer Data Winning the Data Game: Digital Analytics Tactics for Media Groups


Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people

Whitepaper | Digital Marketing Learning to win the talent war: how digital marketing can develop its people


Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
The 2023 B2B Superpowers Index

Whitepaper | Digital Transformation The 2023 B2B Superpowers Index


The 2023 B2B Superpowers Index

The Merkle B2B 2023 Superpowers Index outlines what drives competitive advantage within the business culture and subcultures that are critical to succ...

View resource