AI Scientist: A lucrative role with a shortage of candidates

US companies are raising over $1 billion for the recruitment of AI scientists by 2020. These senior roles have an average annual salary of $314,000, due in part to a global talent shortage. Why are these skills so sought after, what has created the gap between supply and demand, and how can businesses develop their level of AI literacy?

“I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years.” – Andrew Ng

In describing artificial intelligence (AI) as “the new electricity”, renowned computer scientist Andrew Ng made clear how deeply embedded he believes this field will be in every area of business very soon. 

The numbers back up Ng’s prediction: Apple has doubled its number of AI-focused employees since 2014, $5 billion of AI startup funding was made available in 2016, and US companies are harvesting over $1 billion to poach talent in possession of the necessary skills to be an AI scientist.

And yet, this level of investment provides no guarantee of success.

Artificial intelligence is such a vast, nascent, complex area of computer science that even the world’s richest companies are struggling to recruit at the required pace.

Within this article, we will first define what the role of AI scientist entails, before delving into the causes of – and potential solutions to – the global AI talent shortage.

The AI scientist role

The title ‘AI scientist’ has come to encompass a wide variety of responsibilities. In fact, there is still a lack of consensus over exactly what the job titles should be for specific roles in this field, due primarily to the ever-developing nature of AI.

Nonetheless, we can broadly say that an AI scientist is a computer scientist with AI expertise, working towards the development of machines with the cognitive abilities we associate with human beings.

The drive towards an AI-first business culture has entered the public consciousness through digital assistants like Siri and Alexa, but in reality the role of an AI scientist incorporates a lot more work behind the scenes.

An AI scientist may take the lead on development projects, they could focus on research, or they may be data scientists responsible for the quality of the information fed into AI systems. At any level within this hierarchy, an AI scientist will have experience with distributed computational frameworks and will have some academic grounding in statistics or a related field.

AI scientists (particularly AI research scientists) have tended to work in academia until recently; however, there is now a significant ‘brain drain’, with these leading thinkers moving towards lucrative corporate roles.

Their skills are required in more industries than ever before, but also in more departments within individual businesses. Gartner predicts that artificial intelligence will be incorporated into every new product or software by 2020, so we should expect the role of AI scientist to continue its evolution. 

Needless to say, it is a role in high demand, but the supply of talent has been a cause for concern among even the world’s richest companies.

Why is AI talent in short supply?

The New York Times estimates that there are only 10,000 people in the world right now with “the education, experience and talent needed” to develop the AI technologies that businesses are betting on to create a host of new economic opportunities.

There are multiple reasons for this, but a lack of capital investment is certainly not one of them. In the US alone, career advisory platform Paysa has revealed that there are open roles at top companies with the following aggregate net salaries:

• Amazon: $227,769,001
• Google: $130,048,389
• Microsoft: $75,158,057
• Facebook: $38,636,827
• NVIDIA: $34,280,190

Undoubtedly, there is a fierce battle for elite talent and the biggest companies are prepared to pay whatever it takes to win.

At Google-owned DeepMind, which focuses on the development of neural networks to solve AI problems, company records for 2016 showed that their 400 employees were paid an average salary of $345,000 per annum each. There have even been some (almost certainly doomed) suggestions that an NFL-style salary cap will be required to stop the market from spiraling out of control.

So, why are there so few suitable candidates for such lucrative roles – some of which run into five- or even six-figure annual packages?

First of all, artificial intelligence is a very broad phrase that is used as an umbrella term for a variety of disciplines including machine learning, image recognition, and natural language processing. Each of these areas requires a significant level of mathematical skill combined with industry experience (typically 10 years or more), before someone is ready to spearhead projects. As a result, talent gaps are harder to fill when we dive deeper to niche specialisms beyond ‘AI scientist’.

AI is also an industry in flux, with new discoveries and innovations changing the landscape on a frequent basis. This creates a paradox; to become an AI scientist requires years of intense and structured study, but the field of investigation is constantly shifting. It is therefore difficult for academic institutions to keep their courses aligned to the needs of the corporate world.

In essence, the ambitions for AI far outpace the levels of specialist knowledge among even the most experienced computer scientists. As Jeff Dean, the head of Google Brain, put it recently:

“We want to go from thousands of organizations solving machine learning problems to millions.”

In the UK, the Careers and Employability Service predicts a need for more than half a million new workers within the most skilled digital occupations (including AI) before 2022. The number of computer science graduates in the UK would need to increase ten-fold to meet that demand.

This begins to put the challenge in context. With thousands of people capable of performing the role of a senior AI scientist and millions of companies with the potential to profit from the development of sophisticated AI technologies, something has to give.

Furthermore, as businesses as colossal in size as Google make claims to be “AI-first” in everything they do, it is no surprise that organizations are getting creative to solve the AI scientist recruitment problem.

How are companies working to develop AI talent?

In the US, one in three data scientists was born abroad, as companies have gone global in their search for the right talent. Google Brain Toronto, a research facility dedicated to AI, is a manifestation of this desire to hire international talent. Amazon is going further afield, with plans approved for an AI-focused lab in Barcelona to add to a similar facility near the University of Cambridge in England.

None of this investment provides a guaranteed safeguard against the competition, however.

The beauty of an elegantly written AI program is not only that it can open up new and exciting possibilities for all of us, but also that the right people do not always need huge resources behind them to create something revolutionary. Google CEO Sundar Pichai alluded to this in a recent interview when he said:

“You always think there is someone in the Valley, working on something in a garage – something that will be better.”

That may sound affectedly humble from the boss of one of the titans of tech, but it is a genuine sentiment.

The data science community relies on open-source software and collaboration across borders to move algorithms and programs forward. Giant companies like Intel regularly invite proposed solutions to their biggest problems on Kaggle, with the winners often rewarded with large salary offers. These attempts to tap into a decentralized network of data specialists are helpful in the short term, but they still do not drive at the heart of the issue.

The irony is that we don’t have enough people with the skills to create the very AI applications that may end up taking the engineers’ jobs one day, but an increasingly lengthy list of companies are prepared to spend whatever it takes to get there.

What can businesses do today to develop data science skills?

Unsurprisingly, Google is at the vanguard of new, unprecedented initiatives to develop AI expertise that could benefit all businesses. The search giant’s latest initiative, known as ‘AutoML’, has the aim of creating AI that can teach itself to create further AI systems.

That may sound disconcerting, but if successful it could create a wealth of opportunity for mid-sized businesses. Recruiting AI scientists is costly and developing AI scientists requires years of investment, so it would fair to surmise that only the richest companies will acquire this elite talent.

If one of these tech giants should develop an automated technology and open-source the solution, other businesses could tap into this expertise. Said tech giant, in return, would gather the data they crave to fine-tune their algorithms.

This would still not be a panacea for every ill, of course.

There is a base level of data literacy required to make use of these technologies, from data sourcing through to cleaning and processing. Tellingly, a survey produced by the MIT Sloan Management School revealed that 43% of companies report their lack of appropriate analytical skills as a key challenge.

Nonetheless, 63% of companies surveyed by Forbes are now providing in-house data analytics training, which will help employees to get the most out of AI technologies – even if they do not know how to develop the systems themselves.

Paysa estimates that 35% of available AI-related positions require a PhD, but businesses should also focus on hiring mathematics or physics graduates with a bachelor’s degree to fill these positions, as they can be more readily trained in AI specialisms.

Educational resources are increasing both in quality and quantity, too. Andrew Ng has launched Deeplearning.ai in collaboration with Coursera, with the aim of bringing Deep Learning knowledge to a mass audience. There are numerous other MOOCs available on Udacity and EdX, for companies willing to invest in their staff.

There are therefore numerous ways to start improving the level of data literacy at all organizations.

If AI is indeed set to be the new electricity, business leaders should act fast to ensure that their teams are best placed to capitalize.