Can the use of LLMs be detected?
Have you ever been banned from using AI tools like ChatGPT (LLMs) at school? For a while OpenAI and others published experimental software to detect text written by LLMs, but nowadays such tools are hard to find. Is it actually possible to reliably detect the use of an LLM?
My teacher said something like “we have tools to detect the use of LLMs” — I wondered if that’s true, so I looked into it.
How LLMs work in the first place
I only vaguely remember, but the way AI generates text is, simply put, something like “given the previous word(s), the next word will be X” — it outputs based on that kind of pattern. So if you know what prompt was used, you can to some extent predict the output.[^どうやって？]
But if you don’t know the prompt, how would you do it? It probably depends a lot on what data the model was trained on, but for major models is it fairly possible to figure out?
OpenAI
OpenAI previously released software for detecting AI-generated text.
However, it seems the accuracy wasn’t very good, and it probably wasn’t practical.
Quillbot
A lot of people at my previous high school used it. It’s a service that offers an AI detector and a paraphrasing (replacing words with other words) function in the same app — clearly targeting students who try to get assignments done by AI at any cost. They also offer various other tools.
Turnitin
At my school, Keio SFC, there are sometimes messages saying “We’re using Turnitin for this course, so your use of AI will be totally exposed!”.
From what I found, Turnitin provides tools for educational institutions that teachers would want to use, like AI-detection and plagiarism (copy-paste) detection.
wip
This article is a work in progress, and I don’t have deep knowledge of LLM internals, so if anyone knows anything, please let me know~
[＾どうやって？]: Any good references?
0 people clapped 0 times
Related articles
🍳 XR's Use Might Be as a HUD - Let's Cook with AI
XR's Use Might Be as a HUD - Let's Cook with AI
I borrowed an inmo XR — Android-powered XR glasses — from Ukai, familiar from Mitou Junior, and did a little hackathon. After struggling with what to use it for, I built a cooking assistant to enjoy with an AI. I tried using the XR glasses to cook with AI, so I'll introduce that here.
💻 Let's Develop as a Team on GitHub
Let's Develop as a Team on GitHub
For those of you who have registered a GitHub account but don't use it because you're not a developer, here's a brief guide on how to use it.
🤖 Let's Read Google Sheets via API!
Let's Read Google Sheets via API!
This is a re-publication of an article I wrote on another site last year.
It's super easy. I'll...
🗓️ Thoughts on Life Logging Systems
Thoughts on Life Logging Systems
With the advancement of LLMs, it has become easier to verbalize human behavior and analyze conversations with others. However, I felt that none of it quite fit, so I thought about various aspects from both technical and usability perspectives.