Though the general opinion these days is that applied artificial intelligence, in the form of generative AI, will likely improve productivity, that thesis has generally yet to be tested. But one new study of customer service operations at a Fortune 500 firm does provide some early indications of possible impact for CSR operations.
In a study of generative AI at an enterprise software firm, when 5179 customer service agents used a generative AI-based conversational assistant, the tool increased productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low-skilled workers, and minimal impact on experienced and highly skilled workers, the study found.
The study, conducted by professors Erik Brynjolfsson, Danielle Li and Lindsey Raymond, was published by the National Bureau of Economic Research.
“We argue that this occurs because ML systems work by capturing and disseminating the patterns of behavior that characterize the most productive agents,” say the authors. That makes sense. What the engine apparently did was distill best practices and make them consumable by agents who were less skilled.
In other words, “high-skill workers may have less to gain from AI assistance precisely because AI recommendations capture the potentially tacit knowledge embodied in their own behaviors,” they note.
“The AI tool helps newer agents move more quickly down the experience curve: treated agents with two months of tenure perform just as well as untreated agents with over six months of tenure,” they say.
Very few studies of generative AI actually have focused on outcomes in an applied setting, as most prior studies attempted to argue that benefits could be obtained, or should be gotten.
