Flagged by an AI detector? What to do next
Being told your own writing “looks AI-generated” is upsetting, and it happens to honest writers. Here is what is known about how reliable detectors are, and a calm way to respond.
Detectors make mistakes, and their makers say so
- OpenAI withdrew its own detector. When it launched its AI text classifier in January 2023, OpenAI reported that on its test set the tool identified 26% of AI-written text as “likely AI-written” and wrongly labelled human-written text as AI-written 9% of the time. On 20 July 2023 it withdrew the tool, “due to its low rate of accuracy”. (OpenAI)
- Non-native English writers are flagged far more often. A Stanford-led study ran essays written by non-native English speakers for the TOEFL exam through several widely used detectors, which consistently misclassified them as AI-generated, while essays by native speakers were judged accurately. (Liang et al., “GPT detectors are biased against non-native English writers”, published in Patterns, 2023)
- Some universities have switched detectors off. Vanderbilt University disabled Turnitin's AI detector in August 2023. It noted that Turnitin had claimed a 1% false positive rate at launch, and that at that rate around 750 of the 75,000 papers Vanderbilt submitted in 2022 could have been wrongly labelled. (Vanderbilt University)
None of this means a flag is always wrong. It means a detector score on its own isn't evidence of how a piece was written. Evidence of how you wrote it is.
What to do
- Stay calm and ask what was flagged. Ask which tool was used, what it reported, and which passages. Ask what the process is from here and when you can respond.
- Gather your process evidence. Version history from Google Docs or Word (how to find it), your notes and outline, your sources, earlier drafts, and timestamps from emails or uploads. If you wrote the piece in Writing Replay, export the replay.
- Write a short, factual account. When and where you wrote it, roughly how long it took, what tools you used (grammar checkers count), and anything that might look unusual, such as pasting your notes in from another app.
- Offer to talk through the piece. Explaining your argument, your sources and the choices you made is persuasive in a way no file is.
- Use the formal process if you need it. Schools and publishers have policies and appeal routes. Follow them, keep copies of everything, and ask for any decision in writing. This guide isn't legal advice.
For next time
Write where your process is recorded as you go. In Writing Replay, every pause, revision and paste is kept, and your reader can watch it back. A replay can't prove who was typing, and it says so to whoever opens it, but it is exactly the kind of evidence that answers “how did you write this?”. More on showing your process
Sources
- OpenAI, “New AI classifier for indicating AI-written text” (31 January 2023, updated 20 July 2023). openai.com/index/new-ai-classifier-for-indicating-ai-written-text
- Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, James Zou, “GPT detectors are biased against non-native English writers”, Patterns 4(7), 2023. arxiv.org/abs/2304.02819, doi.org/10.1016/j.patter.2023.100779
- Vanderbilt University, “Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector” (16 August 2023). vanderbilt.edu