Can LLM Usage Be Detected?
Have you ever been banned from using AI tools like ChatGPT (LLMs) at school? At one time OpenAI and others released experimental software to detect text written by LLMs, but such tools are hard to find now. Is it actually possible to reliably detect LLM usage?
My teacher at school was like, “We have tools to detect LLM usage,” — but is that true? So I looked into it.
How LLMs Work in the First Place
I’m a bit fuzzy on the details, but the way AI returns text is, simply put, like: “the previous word was this, so the next word is probably that.” So if you know what prompt was used, the output should be somewhat predictable.1
But if you don’t know the prompt, how would you do that? It probably depends a lot on what data the model was trained on, but for major models, is it largely understandable?
OpenAI
OpenAI once released software for AI detection.
However, it seems the accuracy wasn’t great, and it never became practical.
Quillbot
A lot of people used it at my previous high school. It’s a service that offers an AI detector and a paraphrasing feature (replacing words with other words) in the same app — clearly targeting students trying to get AI to do their assignments. They also provide various other tools.
Turnitin
At my school, Keio SFC, there’s sometimes a message saying, “This course uses Turnitin, so your AI use will be totally exposed!”
From what I found, Turnitin offers tools for educational institutions to detect AI use and plagiarism (copy-paste), basically the kinds of tools teachers would like to use.
wip
This article is a work in progress, and I don’t have deep knowledge of LLMs, so I don’t really know much — if anyone knows anything, please tell me~
Footnotes
-
Is there any good reference material? ↩
0 people clapped 0 times
Related articles
💻 Let's Try Team Development on GitHub
Let's Try Team Development on GitHub
For those of you who have registered an account on GitHub but haven't used it because you're not a developer, I've written a brief article explaining how to use it.
🤖 Let's Read Google Sheets via API!
Let's Read Google Sheets via API!
This is a re-publication of an article I wrote on another site last year.
It's super easy. I'll...
🍳 XR's Use May Be Like a HUD — Let's Cook with AI
XR's Use May Be Like a HUD — Let's Cook with AI
I borrowed an inmo XR, an Android-based XR headset, from Ukai (known from Mitou Junior) and did a small hackathon. After struggling to find a use, I built an AI-powered cooking assistant to enjoy cooking with an AI. Here I introduce my attempt to cook with AI using the XR glasses.
🗓️ Thoughts on Life Logging Systems
Thoughts on Life Logging Systems
With the advancement of LLMs, it has become easier to verbalize human behavior and analyze conversations with others. However, I felt that none of it quite fit, so I thought about various aspects from both technical and usability perspectives.