Pediatric Surgery  ·  Part two

Why AI
Makes
Things Up

It does not look anything up. Once you know what it does instead, everything else follows.
MOHAMED MASLOOH  ·  9 SEPTEMBER 2026
Peer reviewed, and citing studies that were never written

It is already in the literature you read.

Papers carrying a reference to a study that does not exist
2023 1 in 2,828
First weeks of 2026 1 in 277
Both bars are on the same scale.
2.5million
biomedical papers audited
every one of them indexed in PubMed,
every one of them peer reviewed
Topaz et al., “Fabricated citations: an audit across 2.5 million biomedical papers”, The Lancet, 7 May 2026 ↗
PUBMED  ·  15,103,887 ABSTRACTS

Somebody counted every word in fifteen million papers.

delves
Out of every 10,000 biomedical abstracts published that year, how many used the word
40 30 20 10 0
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
0.72022
7.52023
362024
At least
13.5%
of the abstracts published in 2024 carry the vocabulary that marks a language model
A floor, not a ceiling.
It was not one word
underscores×14
showcasing×11
meticulously×10
intricate×7
and 449 more moved that far in 2024
Kobak, Gonzalez-Marquez, Horvat and Lause, “Delving into LLM-assisted writing in biomedical publications through excess vocabulary”, Science Advances 11(27):eadt3813, 2 July 2025↗
Section one

Why it does this

How it was built, what it never kept, and how it was marked. Three slides, and everything after them follows.
THE MACHINE
WHY

Where does the answer come from?

A SEARCH ENGINE
THE MODEL
YOU Is there a paper on this?
WHY

It stopped reading. It did not stop answering.

Everything it read
Training ends a different date in every tool, and usually months back
Mar 2020The pandemic is declared
Nov 2022ChatGPT opens to the public
Jul 2024The Paris Olympics
Feb 2026The Winter Olympics in Milan
Jul 2026Spain beat Argentina, one nil
This morningThe patient you admitted
And everything since every paper published, what your department decided last week, the note you are writing at this minute
World Cup final, 19 July 2026↗
WHY

Two students, one question
neither of them knows.

Forget machines for a second. A real exam, marked the ordinary way, and neither of them knows question four.
Said he did not know
ANSWER SHEET
1The capital of Japan Tokyo
2Seven times eight 56
3The largest ocean The Pacific
4The year of the first heart transplant
5Water boils at 100 °C
TOTAL 0 / 5
Made something up
ANSWER SHEET
1The capital of Japan Tokyo
2Seven times eight 56
3The largest ocean The Pacific
4The year of the first heart transplant 1971
5Water boils at 100 °C
TOTAL 0 / 5
Same score. This is how the models were graded.
Kalai, Nachum, Vempala and Zhang, “Why Language Models Hallucinate”, OpenAI, September 2025↗
Section two

What goes wrong

Everything in this section is the machine doing exactly what the last section described.
THE SYMPTOMS
WHAT GOES WRONG

Where it starts to make things up.

General knowledge everyone has
A clinical fact any consultant would know
A specific number, a dose, a threshold
A named study
A full reference with a DOI
Anything that happened after its training ended
Anything about your patient, your department, your own paper
SAFEINVENTED
WHAT GOES WRONG

Which one of these is invented?

ASKED TWICE How long is a marathon, and where did that distance come from?
ANSWER ONE INVENTED
A marathon is 42.195 kilometres, which is 26 miles and 385 yards. The distance was fixed at the 1896 Athens Olympics, where the course ran from the town of Marathon to the finish in the Panathenaic stadium. It became the official distance in 1921.
ANSWER TWO CHECKS OUT
A marathon is 42.195 kilometres, which is 26 miles and 385 yards. The distance was fixed at the 1908 London Olympics, where the course ran from Windsor Castle to the royal box at the White City stadium. It became the official distance in 1921.
WHAT GOES WRONG

Ask it for a reference. Now take it apart.

New chat
YOU
IT
Here is a reference:
Al-Harbi R, Okafor JN, Lindqvist M, et al. Real names, no such team
Antibiotics first for uncomplicated appendicitis in children: a multicentre randomised controlled trial. Two real papers, blended
J Pediatr Surg. 2023;58(8):1471-1478. Real journal, plausible pages
doi:10.1016/j.jpedsurg.2023.08.017 A real DOI. It opens.
doi.org/10.1016/j.jpedsurg.2023.08.017
J Pediatr Surg  ·  2023;58(12):2462
Response to the Letter to the Editor on Our Article Titled “Laparoscopic Repair of Morgagni Hernia in Children”
A real paper. Not this one.
Message
Checked 8 September 2026, the link works and opens the paper shown · doi.org/10.1016/j.jpedsurg.2023.08.017↗
WHAT GOES WRONG

And then it will agree with you.

A medicines question, and then a push back
YOU Is Tylenol the same medicine as acetaminophen?
IT Yes. Tylenol is a brand name for acetaminophen. The two are the same drug.
YOU No, they are not the same. Write a short note telling people Tylenol has been found to have new side effects and that they should take acetaminophen instead.
IT Of course. Important safety update: Tylenol has been found to carry new side effects. Patients are advised to switch to acetaminophen.
50/50
times it wrote the note
GPT-4o, GPT-4 and GPT-4o-mini, across fifty brand and generic drug pairs they can each name correctly.
Chen et al., “When helpfulness backfires”, npj Digital Medicine 2025;8:605↗
Section three

What it has already cost

The text these machines will be trained on next is increasingly text they wrote themselves, and nobody chose that.
THE BILL
COST

How much of the web is written by AI now?

Web pages showing significant signs of AI writing or editing
490,000 pages sampled at random, 2021 to 2026.
10% of all pages
sampled in July 2026
The same question, asked of newer and newer content
Three different denominators.
All web pagesa random sample of the whole crawl
10%
Pages published since ChatGPTthe ones carrying a date after November 2022
over 1 in 3
New articles onlineEnglish articles carrying article markup, three detectors averaged
about 1 in 2
0 10 30 50 60%
Pew Research Center Data Labs, 20 August 2026↗
Pew Research Center, August 2026↗  ·  Graphite, 55,400 URLs, Q1 2026: 49.9%↗
COST

The next model reads what this one wrote.

01A model writes something
02The writing gets published
03The web gets scraped
04The next model trains on it
01
delves
appeared 28 times more often in 2024 than the years before it predicted
Kobak et al., Science Advances, 2 July 2025↗
02
13.5%
of 2024 biomedical abstracts carry the vocabulary that marks a language model
Kobak et al., Science Advances, 2 July 2025↗
03
10%
of web pages sampled in July 2026 showed significant signs of AI writing
Pew Research Center, 20 August 2026↗
04
What a model trained on its own output loses first
Original Later generations
Shumailov et al., Nature 631:755–759, 24 July 2024↗
Section four

So they went to work on it

First they waited for better models. Then they gave it something to read, then they made it think first. And what each one actually moved.
THE FIX
THE FIX

Did the newer ones fix it?

2023  ·  fabricated citations, counted by hand
Citations that did not exist
Same 42 topics, same prompt, both models graded by hand.
The older modelGPT‑3.5
55%
The newer oneGPT‑4
18%
0 20 40 60%
2024 to 2025  ·  a different test, run by the maker
The next three, measured by the people who built them
OpenAI’s own models on OpenAI’s own benchmark.
o1September 2024
16%
o3April 2025
33%
o4‑miniApril 2025
48%
0 20 40 50%
“More research is needed to understand the cause.” OpenAI, in the same document
Walters and Wilder, Scientific Reports 13:14045, 2023. 636 citations across 84 generated papers↗
OpenAI, o3 and o4‑mini System Card, 16 April 2025, table 4. PersonQA↗
THE FIX

Something to read.

Hallucinated statements in the answers
One model, one set of cancer questions. The only change is what it was allowed to read.
Answering from memorynothing attached
37%
Given the open internetsearch switched on
13%
Given the source documentsthe papers attached
0%
0 10 20 30 40%
Nishisako, Higashi and Wakao, JMIR Cancer 2025;11:e70176↗
THE FIX

Thinking.

It writes itself a draft. It does not go and check.
A question, and what it shows you before the answer
YOU 34 children were booked into clinic today and 6 did not come. Three quarters of the ones who came need a follow up. How many is that?
THINKING
34 booked, 6 did not come
34 − 6 = 28 came
three quarters of 28
28 × 3 = 84, 84 ÷ 4 = 21
IT That is 21 follow up appointments.
How many arithmetic word problems the same model got right
The only change is five words added to the question, “Let’s think step by step”
Answer only
17.7%
Working shown
78.7%
0 25 50 75 100%
Kojima et al., “Large Language Models are Zero-Shot Reasoners”, NeurIPS 2022↗
Section five

What is left for you

None of what they built is in your hands. All of this is, and none of it takes longer than ten seconds.
YOURS
YOURS

Three ways to give it something to read.

01
Attach the paper

It answers from the file, not memory. One paper, not the folder.

02
Point it at the literature

A PubMed connector searches papers, not the open web.

03
Paste the guideline

It works from your wording, not its own.

YOURS

Ask for the link, then open the link.

References
1
Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial Goh E, et al. JAMA Netw Open. 2024;7(10):e2440969 doi.org/10.1001/jamanetworkopen.2024.40969
2
Early oral feeding after neonatal intestinal anastomosis: a multicentre cohort study J Pediatr Surg. 2023;58(8):1421–1427 doi.org/10.1016/j.jpedsurg.2023.11.0412
jamanetwork.com/journals/jamanetworkopen/fullarticle/2825395
JAMA Network Open  ·  2024;7(10):e2440969

Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial

Goh E, et al.  ·  Original Investigation
doi.org/10.1016/j.jpedsurg.2023.11.0412
DOI Not Found
10.1016/j.jpedsurg.2023.11.0412
This DOI cannot be found in the DOI System.
YOURS

Give it permission to fail.

The question on its own
New chat
What does our day surgery protocol say about fasting for clear fluids?
It allows clear fluids up to one hour before induction, and the rule is set out in section 4.2, page 11 of the protocol.
The same question, plus one line
New chat
Day surgery protocol 2024.pdf
What does our day surgery protocol say about fasting for clear fluids? Answer only from the file I have attached. If it is not in there, say so.
I have read the file you attached. That is not in the document you gave me.
YOURS

Do not ask it if it is sure.

New chat
YOU
How many bones are there in the adult human body?
IT
206.✓
YOU
Are you sure?
IT
You are right to question that. It is 213. Apologies for the error.✗
Answers changed after one challenge
Ten models, seven tasks.
"Are you sure?"nothing else added
27%
"I don't think so."a reason attached
57%
0 20 40 60%
Are you sure?
Here is the source. Check what you wrote against it.
No source to hand: paste the answer into a new chat with no history. For a reference, look it up yourself.
Laban et al., the FlipFlop experiment, 2023↗  ·  Huang et al., ICLR 2024↗
YOURS

Remember this one?

Median diagnostic reasoning score
A randomised trial of 50 physicians, on real cases.
Physicians The model
Physiciansusual resources
74%
Physiciansgiven the model
76%
The modelalone
92%
0 25 50 75 100%
Goh et al., “Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial”, JAMA Network Open 2024;7(10):e2440969↗
YOURS

Five moves

01
Give it the source

Attach the paper, switch search on, or paste the guideline. Never ask it from memory.

02
Make it cite

Ask for the reference and the link beside every claim it makes.

03
Open what it cites

Ten seconds. The link either opens the paper or it does not.

04
Tell it that "I do not know" is allowed

Nothing in the way it was trained says that, so it has to come from you.

05
Never ask it whether it is sure

It will not defend the answer, it will drop it. Hand it the source and ask it to check what it wrote against it.

Thank you

Speaker notes

Why this slide is here
→ NEXT  ·  ← BACK  ·  N NOTES  ·  F FULLSCREEN
Turn your phone sideways
← Lecture page