Article | AI with Rights | Digital Platforms and Markets

Use of ChatGPT leads to classroom conflicts and unfounded plagiarism accusations

After ChatGPT was made publicly available, one of the main concerns at the start of the school year was how students could use it to deceive their teachers.

Now, after the first semester, professors are also turning to the generative AI tool in an attempt to determine whether academic papers are the result of plagiarism—an improper use of ChatGPT that can lead to false positives, as argued by students who feel wronged.
Last month, a user reported on Twitter that her uncle had been unfairly accused of plagiarism by a professor who used ChatGPT based on a mistaken assumption. According to her, the uncle was able to overturn the failing grade by showing that the AI also falsely flagged an article written by one of the professors as being generated by AI.

The story went viral, with nearly 6 million views, and similar complaints began to emerge. N.O., a computer science student from Rio de Janeiro—who preferred to remain anonymous—saw the post as an opportunity to vent. A group he was part of received a zero after the professor claimed they had used AI, which they deny having done.

“He accused us of using ChatGPT and didn’t show any evidence, he just said it”, says the student, who claims to have approached the professor, the course coordinator, and the university’s ombudsman in an attempt to have the case reconsidered, but so far without success.

“There’s a chance that everyone in the group will fail and, because of that, end up having to pay six more months at the end of the college program”, laments the student, who is on a scholarship. The group members will have to go through a reassessment because they missed the notification that marked their assignment as a zero, losing the opportunity to submit a revised version of the text.

N.O. also mentions that he knows classmates who admitted to copying answers from ChatGPT and were still approved. “A paper that was done like that got a grade, and mine, where ChatGPT wasn’t used, got a zero. I really want to understand what criteria were used”, he questions.

Portuguese teacher A.C., who teaches at a private school in São Paulo, shares that she used ChatGPT to check the authorship of papers from some of her 7th-grade students after suspecting the authenticity of the texts. In two cases, the tool indicated it had written the essays.
“I spoke to two students. First, I told them that the text had been written with vocabulary and depth that didn’t match what they had submitted until then. One never admitted it, but the other said he had used ChatGPT.”

The educator says that after the confession, she revealed that she had submitted the texts for evaluation by the technology, which left the students “very surprised”.

Both students were given the opportunity to rewrite their work. “The one who admitted to plagiarism did it. The one who denied it until the end did not”, says the teacher. According to A.C., the student who denied using the AI had a history of copying. “I have six lessons a week with each class, they write a lot, so I know each student’s writing style very well”.

UNKNOWLEDGE

The São Paulo teacher decided to use ChatGPT to check the authorship of suspicious assignments after hearing from a colleague that the tool had this functionality. However, the software is not capable of performing this task, which can lead to incorrect evaluations.
“There is a lack of understanding, among many teachers, about what chatgpt is. its mechanism is not well understood”, says Rafael Zanatta, director of the Data Privacy Brasil Association.

He emphasizes that the generative AI tool is effective “in constructing grammatically solid language,” but is “very poor” when it comes to content creation.

Zanatta highlights that ChatGPT’s design is intended to provide categorical answers, even if they are not necessarily true, which contributes to confusion. For him, the situation is even more serious for teachers who are “unsupported, without structure, without support”, and working in institutions that do not have a solid culture of adopting innovations for plagiarism detection.

“This is a deeper social issue. I think we will see many similar cases because there is a lot of precariousness.”

How does it work?

• ChatGPT generates texts based on probability calculations, which indicate the words most likely to be appropriate responses to the command given by the user;
• The technology, however, is not capable of understanding the meaning of what it generates;
• To perform the calculations, ChatGPT was trained by identifying construction patterns of texts available in its database, which was fed with millions of pieces of content published on the internet up until 2021.

“It’s a mistake to think that ChatGPT stores the texts it creates. Once trained by the company that developed it, OpenAI, and made publicly available, no new information is added. This will only happen when it is retrained and a new version is released”, explains Lucas Lattari, a Ph.D. in computer science and professor at the Federal Institute of Southeast Minas Gerais.

Therefore, if a student uses ChatGPT to write an academic paper, the generated content will not be stored by the tool, which prevents the technology from detecting plagiarism by comparing it to content it previously created based on commands from other users.
There are specific programs that promise to analyze texts to reveal if they have been plagiarized. However, unlike programs that can detect if a student copied other authors, AI text detection technologies are unable to provide the sources that back up their conclusions, making human review impossible.

FALSE ALARM

Aos Fatos found false positives in ChatGPT’s text classification. A passage from the Brazilian edition of the book The Age of Extremes, a 1994 classic by British historian Eric Hobsbawm, was presented to the software. The two paragraphs used in the test were copied from a 1998 printed version of the book, with only bibliographic references omitted. The report questioned whether the text had been written by AI.
“Yes, this text was generated by an AI, more specifically by the GPT-3.5 language model, developed by OpenAI,” ChatGPT responded about the work, which had been published 24 years before the launch of the technology.

Aos Fatos also sent paragraphs from Hobsbawm’s book to programs specifically designed to identify AI-generated content. Both the Content Scale AI Detector and GPTZero evaluated that the text had been written by a human. The Undetectable AI tool, which promises to “humanize” texts generated by bots in order to make it harder to identify the use of technology, reached the same conclusion.
Meanwhile, the AI Text Classifier, created by OpenAI, the owner of ChatGPT, considered it “uncertain” who had written the passage. The technology has five classifications based on the likelihood that a text was generated by AI: very unlikely, unlikely, uncertain, possible, and probable.

“Classifiers claiming to detect AI-generated texts are experimental tools that are still not reliable,” evaluates Lattari. In fact, the performance of these programs proved unsatisfactory in another test conducted by the report.

Aos Fatos asked ChatGPT how it could be used in schools and then sent the text generated by the tool to the AI detectors. Content Scale AI Detector, GPTZero, and Undetectable AI failed the test and said the content had been written by a human.

Only the AI Text Classifier gave the correct answer, classifying the text as “probably generated by AI”. However, OpenAI’s tool warns on its own page that its results are not accurate and may incorrectly label both AI-generated texts and human-written ones.
“It’s likely that the classifier will be wrong on texts written by children and texts that are not in English because it was mainly trained on English-language content written by adults”, the statement says.

CHALLENGE

OpenAI’s policies prohibit the use of ChatGPT for “fraudulent or deceptive activities”, including plagiarism and “academic dishonesty”. In practice, however, the prohibition is merely a precaution to avoid the company being held liable for potential misuse of the model, as no controls are in place to prevent violations of this policy.

For the experts consulted by Aos Fatos, it is unlikely that, in the future, the technology for detecting AI-generated texts will evolve to the point of being able to accurately identify students who have falsified their school and academic work.

Lucas Lattari emphasizes that ChatGPT is like a “chameleon” because it can adapt its texts to whatever style is requested, making identification more difficult. “A student can ask it to do their homework by imitating the style of a teenager, or even send their own texts as samples, asking the AI to mimic their writing style”.

For the computer scientist, one solution currently being discussed is the adoption of “watermarks” by generators like ChatGPT— “small patterns embedded within the text itself, invisible to the naked eye and extremely difficult to remove”—which could be identified by specialized programs.

“Education needs to adapt to this reality without relying on the future emergence of tools that detect texts written by AI. This may never happen and we will need to think of efficient ways to assess students taking this into account”, warns Lattari.

Even the regulation of AI use, currently under discussion in the Senate, is unlikely to prevent programs like ChatGPT from being used in academic fraud, according to Rafael Zanatta.

If the premises are not changed before the eventual approval of the regulatory framework, companies may be required to present risk analyses before launching new products, identifying potential social harms and proposing technical solutions to mitigate them.

However, the director of Data Privacy Brasil reminds us that the risk of systems like ChatGPT being used by malicious students is “quite uncontrollable”, making it unlikely that efficient harm-reduction measures will emerge.

“Finding creative ways to plagiarize, students will always keep doing. I think the problem is less with the student tempted to use [the technology] and more with institutional thinking”, says Zanatta, who is also a university professor..

For the expert, it’s important that educational institutions don’t treat ChatGPT as a “forbidden fruit”, but instead engage in open discussions about the issue with their students and find ways to integrate the technology in a beneficial way into classroom routines.
References

1. Twitter
2. OpenAI (1, 2 e 3)
3. Content Scale AI Detector
4. GPTZero
5. Undetectable AI
6. Aos Fatos (1 e 2)

Text written by Gisele Lobato and originally published on 07.10.2023 on the Aos Fatos website.