← Back to library

Check an AI-edited paper for dropped or invented citations

u/ringringpssy leaned on ChatGPT plus its research and agent features for a social science paper. It eased writer's block, but rewriting dropped key citations, it invented sources, cited Wikipedia against instructions and botched a comparison of political actors' demands with the final law. For that analysis, the poster found Claude far more reliable.

  • OutcomeAfter any AI rewrite of academic text, compare citations before and after, confirm each source exists and check that claims still match their sources.
  • SetupUnder an hour
  • LevelStarter
  • Creditu/ringringpssy on r/ChatGPT
View original
Works with
ChatGPTClaude

Published Oct 23, 2026Updated Oct 23, 2026AI dry run on sample input: passed Oct 23, 2026Source verified Oct 23, 2026

When should you use it?

  • You use AI to rewrite or polish academic text
  • Your paper cites many sources
  • You asked for scholarly sources only

Not for: Fiction or opinion writing; Drafts you have not cited yet

What do you need, and what do you get?

You need

  • Original draft
  • AI-edited draft
  • Your reference list or source files

You get

  • Before and after citation table
  • List of removed, added or changed citations
  • Claims that need rechecking

How does it work, step by step?

  1. Keep the originalSave your draft before any AI edit.
  2. List citations twiceExtract every citation from both versions.
  3. CompareFlag citations that disappeared, appeared or moved to a different claim.
  4. Verify new sourcesLook up any new source yourself to confirm it exists and says what is claimed.
  5. Check the sentenceRead each cited sentence against the source page.

The prompt

Our own prompt, inspired by u/ringringpssy on r/ChatGPT and written to work in ChatGPT and Claude. Values in lime are placeholders.

Fill in the blanks (optional; the copied prompt uses your values)

Prompt2 variables
Here are two versions of my paper: original {original} and edited {edited}.
List every citation in each version. Show a table of citations removed, added, or attached to a different sentence. For each added citation, say whether it appears in my reference list.
Do not invent or suggest new sources.

Inspired by u/ringringpssy on r/ChatGPT: ChatGPT as research/academic assistant - my experience. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.

What can go wrong?

  • Invented sources can look real; search the title and DOI.
  • Encyclopedia pages may not meet your course's rules.
  • A model can misread a comparison; check it against both documents yourself.

Our take

The poster's experience points to a specific, checkable failure: polished rewrites that silently lose or fabricate citations. That makes the defense mechanical rather than a matter of trusting one model over another. Before any AI edit, save the draft and extract its reference list and every in-text citation marker. After the edit, extract them again and compare; anything removed needs a reason, and anything added needs to be looked up in the library database, not just on the web. For sentences that carry a citation, reread the cited passage, because models often keep the marker while changing the claim. The document comparison that went wrong is a good example of a task to split: have the model list each demand and quote the matching clause of the law side by side, then judge the match yourself. Telling the model to leave citation markers untouched helps but is not a guarantee. The model rankings in the post will change within months; the before-and-after audit will stay useful, and the saved versions protect you if a grader questions a source.

Where the time goes back: A citation diff takes minutes and catches the errors that cost marks.

FAQ

Which model should I use?
The poster found one far better than another for this work, but any model can drop citations; audit either way.
Can AI do the audit?
It can list differences, but confirming sources is your job.

Source and credit

Inspired by u/ringringpssy on r/ChatGPT. We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.

Inspired by
ChatGPT as research/academic assistant - my experience
Creator
u/ringringpssy on r/ChatGPT
Published on
Reddit
Original date
Jun 17, 2026
License
Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
Originally for
Claude
AI dry run
Oct 23, 2026: passed. Dry-run tested by our AI test agent on sample input. PASS: Gates held on desk check; no live sends, bookings or purchases.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails.
Verified
Oct 23, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.

Are you the creator? Claim or correct this page.

More in Career & Learning

All 37 Career & Learning playbooks →

Related playbooks

Career & LearningPrompt✓ AI dry run

Compare your project plan with its fastest possible timeline

Have an AI interview you task by task about a project you keep postponing, estimate how long each task would take with no waiting at all, set that beside your real schedule, name the three largest gaps and whether each comes from waiting, approvals, hesitation or genuine effort, and give you a one-page plan to close the largest gap this week.

Credit: AI Guides (@free_ai_guides)

Minutes
Career & LearningRoutine✓ AI dry run

File assistant explanations into a git notes repo

Turn useful chat explanations into durable notes by having the assistant write a summary file into the right git repository, guided by an AGENTS.md in each repo.

Credit: mindcrime on Hacker News

Under an hour
Career & LearningSkill✓ AI dry run

Learn any topic with a strict Socratic tutor skill

Load Jane Street's open-source tutoring skill into Claude, ChatGPT or a coding agent so that when you ask to be taught something, it first tests what you already know, keeps a 0 to 5 level for each part of the topic, quizzes you one question at a time without handing over answers, re-tests older material by surprise, and only ends the course after a final mixed exam.

Credit: janestreet

Minutes
Career & LearningPrompt✓ AI dry run

Rehearse a negotiation against a counterpart who pushes back

Practice asking for a raise, a better rate or a refund against an AI that argues from the other side's real interests, then get a debrief on where you gave ground.

Credit: u/Ok_Negotiation_2587 on r/ChatGPTPromptGenius

Minutes