Wrong AI Advice May Make You More Confident, Not More Accurate, New Study Finds

Research led by Valerio Capraro at the University of Milan-Bicocca found that people became far more willing to answer difficult questions after receiving AI advice, even when that advice was incorrect. Participants answering obscure movie questions were much less likely to remain silent when unsure, with hesitation dropping from 36–44% without AI to just 3–6% after consulting it.

The problem was that confidence rose while performance moved in the opposite direction. Accuracy fell from 27.5% to only 9.2%, suggesting that AI assistance encouraged participants to trust answers they otherwise might have questioned.

Why It Matters Beyond This Experiment

The findings highlight a growing challenge as AI becomes part of everyday search, homework, and workplace tasks. AI is often viewed as a helpful assistant, but this research suggests it can also lower the mental threshold people use before deciding they are confident enough to respond.

Adding financial incentives made participants slightly more cautious and improved accuracy, yet it did not eliminate the underlying pattern of overconfidence.

A Warning for Education and AI Use

According to Capraro, doubt is good in cases where one needs to check what one does not know. Therefore, AI, which aims at minimising the number of times one is doubtful, may hinder individuals, especially children, from having a questioning attitude towards the information they receive.

Therefore, while making the AI assistants more reliable and efficient, the researchers argue that it is important to make sure that users do not develop the wrong attitudes towards these technologies.

Get the latest tech news on WhatsApp