Research shows workers punished for ignoring AI recommendations, even when correct

Workers are punished for ignoring AI recommendations and making their own decisions, even if the decision leads to a better outcome, according to research by Frankfurt School of Finance and Management and ESMT Berlin. The study found that managers penalised workers through reduced bonuses when they deviated from AI-driven advice, even when human judgment resulted in superior decisions.

The research was conducted by Mirko Kremer, professor of supply chain management at Frankfurt School, Francis de Véricourt, academic director of the Institute for Deep Tech Innovation at ESMT Berlin, and Hossein Nikpayam, postdoc in operations management at ESMT Berlin and Frankfurt School. The researchers aimed to explore how organisations are using AI to streamline processes and whether it leads to unintended consequences.

The study involved a controlled lab experiment where decision-makers chose whether to follow AI recommendations or rely on their own judgment. Managers awarded bonuses based on outcomes, but decision-makers who strayed from AI advice were often punished with lower bonuses, even when their human judgment produced better results. This led employees to rely on AI recommendations out of fear of negative consequences, stifling independent decision-making and critical thinking.

Dr Kremer commented: “While businesses across industries are racing to implement AI algorithms to enhance efficiency and decision-making, it’s crucial to remember that these systems should not be relied upon completely. It is important to remember that human expertise remains essential to navigate complexities, make nuanced judgments, and adapt to unexpected situations that AI might miss—especially in sectors like healthcare, aviation, or autonomous vehicles, where the implications of blindly following AI advice can have huge consequences.”

The researchers suggest that organisations need to strike a better balance between AI-driven processes and human decision-making. They advocate for greater transparency in AI systems, fostering trust in employees’ expertise, and creating open communication channels to encourage feedback on AI recommendations. Without these measures, the reliance on AI could hinder rather than enhance decision-making outcomes.