AI assistants boost productivity but paradoxically risk human deskilling

Recent research has demonstrated the potential productivity gains that can come from incorporating ChatGPT (and presumably other chatbots) into knowledge work.

Wharton Business School Professor Ethan Mollick participated in a study alongside several other social scientists and consultants at the Boston Consulting Group (BCG) who used generative AI to determine whether the tool improved their output.

“For 18 different tasks selected to be realistic samples of the kinds of work done at [BCG], consultants using ChatGPT-4 outperformed those who did not, by a lot,” Mollick wrote in a blog post. “Consultants using AI finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without.”

To test the true impact of AI on knowledge work, the researchers took hundreds of consultants and randomized whether they were allowed to use AI. Both the AI-enabled group and a control group that did not have AI were assigned the same writing, marketing, analytical, persuasiveness and creative tasks. These ranged from writing a press release to performing market segmentation.

AI’s unmistakable impact on creativity

As the research showed, consultants with AI access did significantly better. This was true for every measurement, whether the time it took to complete tasks, the number of tasks completed overall or the quality of the outputs. 

Another significant finding is that AI acts as a skill-leveler. The consultants who scored the worst at the start of the experiment had the biggest jump in their performance (43%) when they used AI. The top consultants still got a boost, but less of one. This suggests AI can help to elevate lower performers closer to the level of top performers.

These results reinforce the findings of a similar study by Stanford and MIT last spring. That experiment looked at the performance of 5,000 customer service agents at a Fortune 500 enterprise software firm who were augmented with gen AI over the course of a year. Agents armed with AI were found to be 14% more productive on average than those who were not, with the least-skilled workers reaping the most benefits, as they were able to complete their work 35% faster.

However, the most highly skilled workers saw little to no benefit from the introduction of AI into their work.

Taken together, these research results could have further workforce implications. For example, companies might find that they can achieve more with the same number of employees, leading to higher revenues. Highly skilled workers could focus on more specialized tasks that AI cannot perform, resulting in a workforce with a broader range of skills.

On the other hand, the added efficiency and productivity that comes with AI augmentation could also set higher performance expectations, possibly causing stress or job dissatisfaction for some. And, this advance could potentially lead to downsizing in some areas.

AI’s potential reach in the job market

This reality is no longer theoretical; the tangible influence of AI on the workforce is evident per a recent report from job placement firm Indeed examining job listings and skills. “AI at Work” offers an in-depth look at how gen AI will impact jobs and the skills needed to perform them.

The report analyzed more than 55 million job postings and 2,600 job skills on Indeed to identify the exposure level. Nearly 20% of jobs were considered “highly” exposed to the impact of gen AI. Highly exposed means that AI could perform 80% or more of the skills required for the position. Another 45% of job listings are moderately exposed, meaning that AI can do between 50% and less than 80% of needed skills.

Can AI perform complex “thinking”?

As this research shows, AI can help people do better work and can perform many work tasks. But is AI up to more complex tasks beyond writing a press release? To find out, Section School, a company focusing on providing education about the best use of AI in business, recently ran a thought experiment on the analytical abilities of current chatbots.

They wanted to determine if a chatbot could handle not just any task, but one thought to be complex, such as feedback from a board of directors.

“Before our most recent board meeting, we asked four AI chatbots to give us feedback on our board slide deck,” Section School reported. The quality of output varied across the chatbots, “but Claude [from Anthropic] was almost as good as our human board. It understood the macroeconomic environment, was appropriately ambitious, and quickly got to third-level implications and big picture opportunities.”

This means AI advisors could one day augment or even partially replace the role of human experts and advisors in evaluating complex decisions, strategies and plans. For now, it is an open question as to whether AI could truly replace human strategic thinking and creativity.

The double-edged sword of AI efficiency

While the results of these various studies suggest that AI is positive for work, a separate paper focusing on the performance of job recruiters found that those who used high-quality AI became lazy, careless and less skilled in their own judgment. When the AI is very good, humans have no reason to work hard and pay attention.

The paper reported: “As AI quality increases, humans have fewer incentives to exert effort and remain attentive, allowing the AI to substitute, rather than augment their performance.” People let AI take over, instead of using it as a tool — essentially falling asleep. This demonstrates that people could readily become overdependent on an AI and fail to exercise their own judgment.

People could move through work on autopilot — just like our cars — in the not-too-distant future. Earlier research on the use of smartphones produced similar results, as reported by The Wall Street Journal. That research suggested that the intellect weakens as the brain grows dependent on phone technology. Likely the same could be said for any information technology where content flows our way without us having to work to learn or discover on our own.

If that’s true, then AI — which increasingly presents content tailored to our specific needs and interests, including at work — could create a dependency that weakens our intellect. This concern was echoed by Daniel Weld, a professor at the University of Washington who studies human-computer interaction, in an Axios article: “I worry that human abilities may atrophy.”

Part of human drive and creativity is testing ourselves against our environment, notably other people. Recently, AI has proven to be more proficient than people in many areas. When people cannot win, it is possible they will simply stop trying.

Seeking balance between humans and AI at work

AI’s influence in the workforce is becoming unmistakable. New studies show that AI-assisted consultants outperform their counterparts, significantly boosting productivity and quality of work. While AI can act as a skill-leveler for lower performers, it can also foster dependency, leading to human deskilling.

As AI integrates further into job roles, the double-edged sword of its efficiency becomes more apparent. Companies must tread carefully, leveraging AI’s strengths without compromising human skills and judgment.

Overall, it is clear AI can boost productivity if used judiciously, but organizations must be cautious to not implement the technology in ways that degrade human capabilities. Finding the best division of labor between people and AI is crucial to maximize human engagement and leverage the strengths of each.

Gary Grossman is the SVP of the technology practice at Edelman and global lead of the Edelman AI Center of Excellence.