What AI cannot do for a writer is set out, surprisingly plainly, by the companies that make it. OpenAI's help page lists "fabricated quotes, studies, citations" among the errors ChatGPT can make, and says a model may be unable to reach a page behind a paywall (OpenAI, n.d.-a). Google says Gemini "can hallucinate" (Google, n.d.). Anthropic tells users not to treat Claude as "a singular source of truth" (Anthropic, n.d.-a).
EdCitation publishes this guide. It sets out seven limitations of generative AI tools in academic writing, each from a published study or the maker's own documentation, all read on 26 September 2026. It is not a case against AI tools, only a list of the jobs they cannot do for you.
Several of those jobs are what EdCitation's free tools were built for. Verify references looks every entry up in the publisher's record, Find sources finds published work for a claim, Cite a source builds the reference from the record, and Check your paper reads your assignment's instructions into a checklist. EdCitation never writes any part of a paper.
What can't generative AI do for a writer?
Generative AI cannot do seven things a writer needs, and the table puts each beside what to do instead.
| Task | What AI tools do | What they cannot do | What to do instead |
|---|---|---|---|
| Give a reference | Write text in the shape of a reference | Guarantee the paper exists or says what is claimed | Look each one up: Verify references |
| Summarise a source | Summarise what they can reach or recall | Report an article they could not read, such as one behind a paywall | Read the source; find a readable copy with Find sources |
| Choose sources | Suggest well-known works | Judge what your field and your marker count as authoritative | Your reading list, your librarian, and filters for kind and years |
| Argue and write | Offer ideas and fluent sentences | Hold your argument, your voice, or your memory of what you wrote | Write it yourself; keep notes and drafts |
| Answer for the work | Produce text | Take responsibility for it | You answer for every sentence; disclose use where asked |
| Follow the brief | Follow what you paste in | Know your module's rules, including its rules on AI | Read the brief into a checklist with Check your paper |
| Be current | Answer from training data, or search | Know events after the cutoff unless a search finds them | Check dates and retractions with Cite a source |
Can an AI tool guarantee that a reference exists?
No. A language model writes a reference the way it writes any sentence, and nothing in that process checks a publisher's record. Why AI tools invent references explains the mechanism and how to check a reference is real the hand check; here are the measurements.
Walters and Wilder (2023) had ChatGPT write short literature reviews on 42 topics and checked all 636 references: 55% of those from GPT-3.5 and 18% from GPT-4 were fabricated, and many of the real ones carried substantive errors. Two years later, Linardon et al. (2025) asked GPT-4o for six mental health reviews and found 35 of 176 citations (19.9%) fabricated, with errors, most often a wrong or invalid DOI, in 45.4% of the real ones. Fabrication ran at 6% for depression and about 28% to 29% for two less-studied disorders: the less familiar the topic, the worse the list.
For the tool you use, see the guides on ChatGPT, Claude and Gemini.
What Verify references returned for five entries
On 26 September 2026 we pasted five references into EdCitation's Verify references: three real papers cited here, and two we invented in the patterns AI tools produce. One credited a paper nobody wrote to the two real authors of a Science Advances study, with a DOI in that journal's format. The other gave an invented 2022 article on first-year writing a real DOI from Computers & Education.
The three real papers came back verified, each with "The DOI resolves to this record and the title matches." The first invented entry came back not found: "No publisher's or registry's record matches this reference. No registry holds the DOI 10.1126/sciadv.adq7712." The second came back doubtful ("check this"): "The DOI is real, but it belongs to a different article from the one described." That DOI belongs to a paper by Kong and Lai on a computational thinking test for primary education. None came back "could not check" in this run; when one does, the tool says so and never reports it as not found.
Does an AI tool know what a paywalled article says?
Not unless it has read it, and its maker says it may not be able to. OpenAI (n.d.-a) lists "lack of access" among ChatGPT's limits: a model may not reach a website because of technical problems, paywalls or a site's robots.txt settings. A tool that cannot open an article can still describe it, from its training data or from other pages that mention it, and the description will read as confidently as one based on the full text.
Why a summary is not a reading
Anthropic (n.d.-a) asks users to go back to the sources Claude cites, because the original pages may hold context or detail its summary left out. Even when the paper was real, Walters and Wilder (2023) found substantive errors in 24% of the citations GPT-4 gave. If a claim in your paper rests on a source, read the source. Find sources has an open access filter for works anyone can read in full; your library holds most of the rest.
Can AI judge which source is authoritative for your field?
No. In the largest test we found, the model leaned towards the most-cited works, which are not always the sources your field or your marker would choose. Algaba et al. (2025) asked GPT-4 to suggest references for 166 AI conference papers published after its training cutoff: about 65% of its suggestions existed, and those that did had a median of 1,326 more citations than the references the authors had actually used. The authors call this a heightened citation bias, and it held after controlling for year, venue and other factors.
Famous is not the same as right for your question
OpenAI (n.d.-a) warns that ChatGPT may "misrepresent the weight of scientific consensus". Authority in a field is local: the clinical guideline your nursing module requires, the edition your reading list names, the recent study that replaced the famous one. Start from your reading list and your subject librarian; in Find sources, set the kind of source and the years before you search, and use Cite a source, which screens each source for retraction.
Can an AI tool hold your argument and your voice?
It cannot hold them for you, and the early evidence suggests leaning on it weakens your own hold. Doshi and Hauser (2024) gave some writers story ideas from a language model: their stories were rated more creative and better written, especially among less creative writers, but they were more similar to one another than stories written without it.
What happens to what you remember
Kosmyna et al. (2025) followed 54 participants writing essays over several sessions, with a language model, with a search engine, or with neither. The group that used the model reported the lowest sense of ownership of their essays and had trouble quoting what they had written. The study is a preprint, not yet peer reviewed, so read it as a warning rather than a settled finding. The guide on writing in your own voice covers keeping notes and drafts as a record.
Who is responsible for what an AI tool wrote?
You are. The Committee on Publication Ethics (2023) says AI tools cannot be authors because they cannot take responsibility for a paper, and that authors are fully responsible for everything in it, including the parts an AI tool produced. Universities say the same to students: the University of Toronto (n.d.) reminds students that they are responsible for avoiding an academic offence, and notes that AI tools are known to produce factual errors and inaccurate citations.
Responsibility and disclosure
An invented reference in your list is your invented reference, whoever typed it. Many journals and universities also ask you to say how you used an AI tool, as set out in should you disclose AI use, and how and how to declare AI use in an assignment.
Does an AI tool know your assignment's rules?
No. It knows what you paste in, and your rules live in your brief, your rubric and your module's own policy on AI. University College London (n.d.) sorts assessments into three categories, from AI not allowed at all to AI as an integral part, but leaves the final decision to the member of staff who sets the assessment, and tells students that instructions in the brief take precedence over its general guidance. The University of Toronto (n.d.) treats AI used against an instructor's instructions as an unauthorised aid.
A chatbot cannot tell you which category your essay is in. EdCitation's free Check your paper reads an assignment's instructions into a checklist: word limits for each part, required headings, how many sources and how recent, the citation style, font, spacing and margins, each rule shown with the sentence it came from. More on reading a brief is in how to read an assignment brief and rubric.
Is an AI tool current past its training data?
No. A model knows its training data up to a cutoff date; anything later reaches it only through a search, as OpenAI (n.d.-a) says. The makers publish those dates:
- OpenAI (n.d.-b) gives GPT-5.6 Sol a knowledge cutoff of 16 February 2026.
- Anthropic (n.d.-b) says Claude Opus 5.5 was trained on data up to June 2026, and that its models may not know of events after their cutoff dates.
- Google DeepMind (2026) gives Gemini 3.6 Flash a cutoff of March 2026, but says that in some domains its knowledge may stop at January 2025.
So a cutoff is not a single line: a model can know one field to last spring and another only to the year before.
What search changes, and what it does not
With web search on, a tool can reach newer pages, and OpenAI (n.d.-a) says search is on by default for its models. But search brings back the first three limits: pages it cannot open, sources chosen by a ranking you cannot see, and a summary you still have to check. A retraction issued after the cutoff is unknown to the model unless a search finds the notice; the guide on retracted papers in your reference list explains how to catch one.
How do you use AI tools without handing them these jobs?
Use them where they help, such as brainstorming search words, and keep the seven jobs above for yourself and for tools that look things up:
- Read the brief first, and find the module's rule on AI. Check your paper turns the brief into a checklist, each rule with its sentence.
- Take sources from a search of published records rather than from a chatbot's list. Find sources takes a topic or the claim a sentence needs.
- Read each source you cite, or at least the part your sentence relies on.
- Build each reference from the record with Cite a source, which also screens for retraction.
- Write the argument yourself, and keep your notes and drafts.
- Paste the finished list into Verify references, and fix or replace anything marked "check this" or not found.
- Disclose any AI use your university or journal asks you to.
Which EdCitation tools do the jobs AI tools cannot?
For the first limit, a reference that may not exist, EdCitation's Verify references is the best tool, because it does the one thing a chatbot cannot: it looks each entry up in the publisher's record instead of writing one. In the run above, the invented entry with a borrowed DOI would pass a quick glance at the link; the tool caught it by comparing the record's title with the one described. Free with no account, it takes a pasted list or an uploaded paper, marks each reference verified, "check this" or not found, flags retracted papers, and never shows "could not check" as "not found". With the whole paper uploaded, it also matches citations in the text to the list.
Find sources searches about 300 million published works by topic or by claim, with filters for kind of source, years and open access. Our search on 26 September 2026 for "ChatGPT fabricated citations do not exist" put two 2026 pieces on the problem among its first ten results, one in The Lancet ("Fabricated citations at scale: from detection to prevention") and one in the Journal of the American Veterinary Medical Association, beside unrelated titles that shared the words "do not exist", such as a philosophy chapter by Stephen Yablo. Every result was a published record; choosing among them was still our job, which is the third limit again.
Cite a source, free, builds the reference from a DOI, a title, a PubMed ID, an ISBN or a web address, in six styles. From the DOI of Doshi and Hauser's paper it returned, in APA 7:
Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), Article eadn5290. https://doi.org/10.1126/sciadv.adn5290
In the text: (Doshi & Hauser, 2024). Check your paper reads the brief free; judging the paper against that brief is Theoretics QA, part of Max at $24 a month, which quotes the paper and never rewrites it. Pricing sets out the plans. Whatever the plan, EdCitation never writes a sentence of your paper.
Quick questions
Can ChatGPT check whether its own references are real?
No. Checking means looking the reference up in a publisher's record, and a chatbot answering from memory writes a reply rather than looking anything up. EdCitation's free Verify references does the look-up for a whole list.
What is ChatGPT's knowledge cutoff?
It depends on the model. OpenAI's API documentation gives GPT-5.6 Sol a knowledge cutoff of 16 February 2026, and its help page says answers do not include later events unless search or another tool is used.
Can AI read paywalled articles?
Not reliably. OpenAI says its models may be unable to reach a page because of paywalls or a site's robots.txt settings, so a summary of such an article may not come from the article at all.
Is it my fault if an AI tool invented a reference in my paper?
Yes, in the eyes of journals and universities. The Committee on Publication Ethics holds authors fully responsible for everything in a paper, including what an AI tool produced, and universities take the same line with students.
Can AI tell me whether I am allowed to use AI on an assignment?
No. That rule is in your brief or your module's policy, and at UCL the member of staff who sets the assessment makes the final decision. Read the brief, and ask the module leader when it is silent.
References
- Algaba, A., Mazijn, C., Holst, V., Tori, F., Wenmackers, S., & Ginis, V. (2025). Large language models reflect human citation patterns with a heightened citation bias. In L. Chiruzzo, A. Ritter, & L. Wang (Eds.), Findings of the Association for Computational Linguistics: NAACL 2025 (pp. 6844–6879). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-naacl.381
- Anthropic. (n.d.-a). Claude is providing incorrect or misleading responses. What's going on? Claude Help Center. https://support.claude.com/en/articles/8525154-claude-is-providing-incorrect-or-misleading-responses-what-s-going-on
- Anthropic. (n.d.-b). How up-to-date is Claude's training data? Claude Help Center. https://support.claude.com/en/articles/8114494-how-up-to-date-is-claude-s-training-data
- Committee on Publication Ethics. (2023). Authorship and AI tools [Position statement]. https://doi.org/10.24318/cCVRZBms
- Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), Article eadn5290. https://doi.org/10.1126/sciadv.adn5290
- Google. (n.d.). Learn about responses from Gemini Apps. Gemini Apps Help. https://support.google.com/gemini/answer/16279220
- Google DeepMind. (2026, July 21). Gemini 3.6 Flash model card. https://deepmind.google/models/model-cards/gemini-3-6-flash/
- Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing task [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2506.08872
- Linardon, J., Jarman, H. K., McClure, Z., Anderson, C., Liu, C., & Messer, M. (2025). Influence of topic familiarity and prompt specificity on citation fabrication in mental health research using large language models: Experimental study. JMIR Mental Health, 12, Article e80371. https://doi.org/10.2196/80371
- OpenAI. (n.d.-a). Does ChatGPT tell the truth? OpenAI Help Center. https://help.openai.com/en/articles/8313428-does-chatgpt-tell-the-truth
- OpenAI. (n.d.-b). GPT-5.6 Sol. OpenAI API documentation. https://developers.openai.com/api/docs/models/gpt-5.6-sol
- University College London. (n.d.). Three categories of GenAI use in assessment. Teaching & Learning. https://www.ucl.ac.uk/teaching-learning/generative-ai-hub/three-categories-genai-use-assessment
- University of Toronto. (n.d.). Using ChatGPT or other generative AI tool on a marked assessment. Academic Integrity. https://www.academicintegrity.utoronto.ca/perils-and-pitfalls/using-chatgpt-or-other-ai-tool-on-a-marked-assessment/
- Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, Article 14045. https://doi.org/10.1038/s41598-023-41032-5