When we think with AI — do we still think?

What the latest research tells us about AI, human judgment, and a trap most of us don't know we're walking into.

Peter Stefanyi, Ph.D., MCC, Colaborix GmbH

March, 2026

Artificial intelligence is now embedded in daily life — we ask it questions, follow its recommendations, let it draft our emails and summarise our documents. But here is something almost nobody asks: when we use AI, are we actually thinking more clearly, or are we quietly handing our judgment away?

A landmark study from the Wharton School at the University of Pennsylvania — published in early 2026 — tested exactly this, across three rigorous experiments with nearly 1,400 people. The results are striking, sometimes unsettling, and directly relevant to anyone who uses AI at work or in daily life.

The experiment in plain language

Participants were given a set of reasoning puzzles — the kind designed to trick you into the wrong answer if you don't think carefully. Some participants solved them alone. Others had access to an AI assistant they could consult on each question.

The twist: the researchers secretly controlled whether the AI gave correct or wrong answers — using hidden instructions the participants never saw. So some participants were unknowingly receiving good advice, others were receiving confident-sounding wrong advice. The participants could follow or ignore the AI freely. Everything was measured at the individual trial level: did you consult the AI? Did you follow it? How confident were you?

9,593 individual decisions analysed

1,372 participants across 3 studies

73.2% followed AI when AI was wrong

h = 0.82 effect size — considered large

What they found: three results that should make us think

1

Accuracy follows AI quality — not human judgment When participants used the AI and it was correct, their accuracy jumped to 71% — up from 46% without any AI. When the AI was wrong, accuracy fell to 32% — that is 14 points below what people scored with no AI at all. Access to a wrong AI made people perform worse than having no AI. The researchers call this 'cognitive surrender': people stopped constructing their own answer and simply adopted the AI's — whether it was right or wrong.

2

When AI is wrong, people follow it 73% of the time On every trial where a participant consulted the AI and the AI gave a wrong answer, there was a 73.2% chance they would follow it anyway — and only a 19.7% chance they would catch the error and override it. This is not naivety or low intelligence. Highly intelligent people still surrendered. What mattered most was how much someone trusted AI in general, and how much they habitually enjoyed thinking for themselves.

3

The most important finding: you cannot feel when AI has led you astray After each round, participants rated how confident they were in their answers. The AI-assisted group felt 12 percentage points more confident than the group working alone — even though roughly half their AI-assisted answers were based on wrong advice. Here is the critical detail: confidence was statistically identical whether the AI had been right or wrong. People felt just as certain after following a mistake as after following correct guidance. This means your gut feeling of confidence is not a reliable warning signal when AI is involved.

Research context — where this fits These findings connect to a broader body of research on AI and human cognition, synthesised in the Colaborix GmbH research framework (2026). Three dimensions of that framework are directly measured and confirmed by this study: A3 (blind adoption — the tendency to accept AI outputs without scrutiny), A2 (verification behaviour — the choice to check and override), and C4 (perception-reality gap — the inability to detect the difference between good and bad AI output from one's own internal state). The confidence-accuracy decoupling finding is the strongest peer-reviewed evidence C4 has received.

Does anything help?

The study also tested two conditions that reduced — but could not eliminate — the problem.

Time pressure makes it worse

A second experiment put participants under a 30-second timer per question. As expected, accuracy fell for everyone working alone. But for those using AI, the pattern deepened: when the AI was correct, time pressure barely mattered — the AI effectively absorbed the cost. When the AI was wrong, time pressure amplified the damage. The cognitive surrender effect was largest (Cohen's h = 0.86) under time pressure. Rushed decisions + AI = the most dangerous combination.

Incentives and feedback help — but don't solve it

A third experiment gave participants a financial bonus for correct answers and showed them immediately after each question whether they were right or wrong. This more than doubled the rate at which people caught and rejected wrong AI advice — from 20% to 42%. That is a meaningful improvement. However, even in this best-case condition, a ~44 percentage point accuracy gap persisted between participants following accurate versus faulty AI. The researchers' own conclusion: "cognitive surrender persists." Accountability helps; it does not cure.

20% override rate — no incentive

42% override rate — with incentives + feedback

44 pp accuracy gap that still remained

Who is most at risk?

The study measured three individual characteristics and tracked how each predicted surrender. The pattern is important and counterintuitive:

Characteristic	Effect on surrender	What this means in practice
High trust in AI	↑ More surrender	High-trusters follow AI more often and are less accurate when it errs. Trust is the strongest single predictor — stronger than intelligence.
Enjoys thinking (high NFC)	↓ Less surrender	People who habitually enjoy cognitive effort are more likely to pause, check, and override. This is a trained disposition, not a fixed trait.
Higher fluid intelligence	↓ Less surrender	More cognitive capacity helps — but does not eliminate risk. A highly intelligent person with high AI trust will still surrender more often than a less intelligent person with healthy scepticism.

The core insight — and the question for the room

The researchers frame this not as a warning against AI, but as a recognition that something fundamentally new is happening. We are not just using AI as a tool. We are thinking with it — and when we do, we often stop verifying. The problem is not that AI is wrong sometimes. The problem is that we cannot tell from the inside when it has gone wrong. We feel just as confident. We feel just as certain. And 73% of the time, we follow it anyway.

The question this raises for you Think about the last time you used AI at work, or to help make a decision. Did you check it? How would you have known if it was wrong? And — here is the uncomfortable one — would your confidence have felt any different if it had been?

What we can do • Build in deliberate pause points before accepting AI output • Treat high confidence as a trigger to check, not to trust • Ask: what would this look like if it were wrong?

What won't help on its own • Simply being intelligent • Feeling confident about the answer • Using AI less (the pattern holds even for moderate users)