AI reduces intellectual humility: more confidence, less accuracy

What happened?

Researchers from the University of Milano-Bicocca, the École Normale Supérieure, and Sapienza University of Rome published a study titled “AI advice suppresses people’s willingness to say ‘I don’t know’, even when the advice is wrong and accuracy is incentivized”. In it, they demonstrated that when participants had access to advice from a large language model (LLM), the willingness to admit ignorance plummeted from 44% to 3%, while accuracy fell from 27% to 9%. Despite worse results, confidence in answers increased from 30% to 76%. This phenomenon, documented in a The Register article, reveals a dangerous overconfidence bias induced by AI, persisting even when users are warned that the model may be wrong.

Experiment design

The researchers selected questions about visual details from movies (such as the color of the uniform in Bend It Like Beckham or Monica's vehicle in Like a Cat on a Highway), knowing that LLMs often fail on such data. They used the Step 3.5 Flash model, which gave wrong answers in most cases, to ensure that any reduction in judgment was not due to sensible delegation to a reliable tool. Additionally, they tested recent frontier models (GPT-5.5, Claude Sonnet 4.6, Gemini 3.5 Flash), which failed on the vehicle question but got other details right, confirming the problem is widespread. The experiment included a control group without AI access and an experimental group that could consult the model. To evaluate the effect of incentives, it was repeated with monetary rewards for correct answers. Even then, willingness to admit ignorance only rose to 8% and accuracy to 16%, far below the initial 44% and 27%. This indicates the bias is robust and not easily corrected by external motivation.

Why it matters

The key finding is that AI not only fails to improve accuracy but suppresses intellectual humility. People stop questioning and accept incorrect answers with high confidence. This has profound implications for education, decision-making, and opinion formation. Valerio Capraro, co-author of the study, noted: “For humans, the ability to say ‘I don’t know’ is very important because it represents the recognition of the limits of our knowledge.” AI, by providing easy answers, eliminates that reflective process. This phenomenon echoes classic cognitive biases like overconfidence or authority bias, but amplified by the apparent infallibility of technology. Historically, the advent of the calculator did not suppress mental estimation ability, but generative AI affects epistemic confidence more deeply.

Expected consequences

• Education: Children growing up with AI assistants may not develop the critical thinking needed to evaluate information. Previous studies on search engine use already showed reduced memory and reasoning, but AI worsens the problem by offering direct answers without the need to filter sources.
• Workplaces: Professionals who blindly trust AI could make erroneous decisions with overconfidence. For example, in medical diagnosis or legal advice, where accuracy is critical, overconfidence could have serious consequences. A 2025 Gartner report estimates that 40% of companies already use generative AI in key processes, and lack of human verification could increase errors.
• Society: The spread of incorrect information could accelerate if people neither verify nor doubt AI-generated answers. During the COVID-19 pandemic, misinformation spread rapidly; with AI, the risk is that falsehoods are presented with an appearance of authority, reducing public skepticism.

Even with monetary incentives, improvement was marginal: willingness to admit ignorance rose to 8% and accuracy to 16%, still far below baseline levels. This suggests the problem is not just cognitive laziness, but a bias rooted in how we process automated information.

What readers should know

The problem is not just with the models, but with how we use them. Researchers recommend AI literacy from an early age and educational policies that foster healthy skepticism. Capraro warned: “The incentives of model providers are not aligned with accuracy; the solution must come from education.” Compared to the introduction of the printing press, which democratized knowledge but also spread errors, AI requires a new social contract: teaching users to doubt machines. Initiatives like UNESCO's to integrate digital critical thinking into school curricula are a step in the right direction. Meanwhile, readers should remember that AI is a fallible tool and that intellectual humility remains an invaluable asset.

“The ability to say ‘I don’t know’ is fundamental to human knowledge. AI, by offering easy answers, can erode that ability.”

In summary, AI can be a powerful tool, but its uncritical use can weaken our ability to recognize uncertainty. Intellectual humility is an asset we must preserve. As a society, we need to rethink our relationship with technology to avoid blindly delegating our capacity for judgment.