Peer review: a tool for detecting AI-generated content?

In today's academic environment, the integration of artificial intelligence tools has become commonplace. Technology, particularly that relating to automatic text generation such as ChatGPT, brings a new dimension to traditional student assessment methods. With these advances, the authenticity of work submitted by students is put to the test, requiring adaptive assessment methods.

Peer assessment is a proven method that, in addition to its pedagogical benefits, could offer a unique perspective in detecting inauthentic content. This article explores how peer assessment can be used to identify work that may not be the product of the student's original effort.

ChallengeMe is at the heart of this thinking, having integrated peer review into multiple learning contexts. We invite you to discover the potential of this approach to the challenges posed by artificial intelligence technology in the field of education.

ChatGPT and AI-generated content

At the dawn of an era dominated by artificial intelligence, it is becoming essential to understand the tools that are redefining the interactions between technology and humans. ChatGPT, developed by OpenAI, is one such innovative tool that has the power to simulate human conversations, write essays, poems, or even generate computer code. Based on a sophisticated natural language processing architecture, ChatGPT learns massive amounts of text to produce content that can often seem indistinguishable from that written by humans.

The content generated by ChatGPT and similar language models presents a double-edged sword. On the one hand, it offers immense opportunities for education, student support and research. On the other hand, it raises questions about the validity and authenticity of academic work. These works, when generated by AI, can escape standard detection methods that look for signs of plagiarism or other forms of cheating already known.

The nuance lies in the fact that AI-generated content will not be marked as plagiarized insofar as it is created on demand and does not directly copy existing sources. It therefore requires a new approach to its identification, one that looks not only at the content itself, but also at the way it is produced and the presumed author's understanding of it.

In this context, peer review takes on new importance. It focuses not only on assessing the final product, but also on understanding the thinking process and the knowledge behind it. In the next chapter, we'll take a closer look at how peer review can serve as a complement, or even an alternative, to traditional detection methods in the face of the challenge posed by AI-generated content.

Why traditional plagiarism detection tools are not enough

Plagiarism detection tools have become indispensable in educational institutions to ensure academic integrity. However, these systems are primarily designed to identify direct matches between submitted text and an existing database of published work or student archives. When it comes to AI-generated content, these tools come up against significant limitations, as the text produced does not necessarily reflect direct similarities or duplicated content that is traditionally considered plagiarism.

Content created by tools such as ChatGPT may be unique and have no precedent in plagiarism databases, rendering detection by these means largely ineffective. What's more, these tools often lack the sophistication to assess the context and nuances that characterize plagiarism in AI-generated work. They are not equipped to analyze the depth of understanding or originality of the student's thought process, which is essential for judging the authenticity of an academic work.

Faced with these challenges, it becomes clear that a more nuanced, human-centered approach is needed. This approach should take into account not only the final product, but also the creative process: how the student arrived at these conclusions, what understanding he or she has of the subject, and whether this understanding is reflected in the work presented. In other words, focus on those aspects of intellectual creation that cannot easily be reproduced or imitated by a machine.

This is where peer evaluation comes in as an essential complement. It enables a form of analysis and critique that goes beyond algorithms to the very essence of learning and intellectual exchange. The next chapter will look at how peer assessment can be adapted to meet these challenges, highlighting the importance of human interaction and critical reflection in the assessment process.

Peer review as a detection mechanism?

Peer review is a well-established pedagogical method that engages students in a process of critical examination of their peers' work. This practice, widespread in educational contexts, is particularly effective in developing critical skills, deep thinking and collaborative learning. But beyond these pedagogical benefits, peer review also offers unexplored potential as a mechanism for detecting a new kind of challenge: AI-generated content.

Unlike plagiarism detection software, which analyzes texts in a binary fashion, peers bring contextual understanding and sensitivity to linguistic nuances to the evaluation. By immersing themselves in the analysis of a text, peers are able to notice subtle irregularities such as unusual turns of phrase, odd transitions or a lack of personality in the writing, which are often indicators of inauthentic content.

Humans, with their capacity for abstraction and reasoning, can ask critical questions that test the authenticity of content. For example, when evaluating an argument, a student may be asked about the sources of his ideas, or how he constructed his conclusions. These interactions provide a depth of analysis that plagiarism detection software simply can't replicate.

In addition, peer review encourages students to reflect on their own writing process and recognize the value of original content and independent thinking. This creates a culture of academic integrity where students are less likely to resort to methods such as using AI-generated content for their work.

Establishing authenticity criteria

To refine the effectiveness of peer review in detecting AI-generated content, the establishment of clear authenticity criteria is paramount. These criteria serve as a guide for reviewers, enabling them to judge the quality and originality of submitted work. Here is a non-exhaustive list of criteria to consider:

Narrative and argumentative coherence: The text must demonstrate a logical sequence and a natural progression of ideas. AI-generated texts can often veer off on tangents or lose the main thread.
Personality and unique style: Every author has a stylistic "fingerprint". AI-generated texts can lack this personal touch, resulting in content that seems generic or impersonal.
Relevant references and citations: Works should include contextually appropriate references that support the argument. IA content may fail to incorporate these elements in a meaningful way.
Depth of critical reflection: Authentic work often shows signs of deep analysis and serious contemplation. AI-generated texts can lack this depth, simply repeating ideas without real examination.
Response to specific feedback: A submitted work must be able to integrate previous feedback and show how the author has dealt with specific criticisms.
Complexity of syntactic constructions: Human authors use a variety of complex sentence structures and linguistic nuances that AI may find difficult to imitate consistently.
Natural errors and learning: Human errors, when they occur, often follow a pattern of learning and improvement. Conversely, AI errors can be more random or systemic.

You can add a 4-level rating scale to each criterion, for example:

Narrative and argumentative coherence

Level 1 (Weak): The text lacks coherence, with ideas that seem random or unconnected.
Level 2 (Basic): The text shows attempts at coherence, but the logic can be difficult to follow at times.
Level 3 (Good): The text presents a clear logical sequence with well-connected arguments, despite a few gaps.
Level 4 (Excellent): The text demonstrates excellent coherence, with a fluid, natural progression of ideas and well-structured arguments.

Unique personality and style

Level 1 (Low): The writing style is generic, with no personal or distinctive characteristics.
Level 2 (Basic): Signs of personal style are visible, but are not constantly maintained throughout the text.
Level 3 (Good): The writing style is clearly personal, with a distinct voice that stands out in the majority of the text.
Level 4 (Excellent): The style is not only unique and personal, but also adds to the argument and engagement of the reader.

Depth of critical thinking

Level 1 (Low): The text repeats ideas without critical engagement or in-depth analysis.
Level 2 (Basic): Attempts at critical analysis are present, but remain superficial.
Level 3 (Good): The text shows good critical analysis, with relevant and well-developed reflections.
Level 4 (Excellent): The text demonstrates exceptional critical thinking, with deep insights and a nuanced understanding of the subjects covered

These criteria, when applied by trained and attentive peers, are formidably effective against AI-generated content. They exploit the human capacity to detect authenticity and creativity, aspects often lacking in machine-generated work. Peer evaluation, enriched by these criteria, thus becomes not only a collaborative learning tool, but also a bulwark against the intrusion of non-authentic content into the academic world.

Training students for effective assessment

To ensure that students are able to carry out authenticity-oriented peer reviews, appropriate training may be of interest. For example

Understanding AI: Start by explaining how AI content generation tools work, their strengths and limitations.
Authenticity Criteria: Provide detailed evaluation grids based on established authenticity criteria, with concrete examples for each quality level.
Practical workshops: Organize group assessment sessions where students can practice assessing anonymized texts, with a debriefing to discuss the assessments.
Constructive feedback: Teach students how to formulate constructive feedback, which helps the author understand areas for improvement.
Using Real Cases: Use case studies and real-life examples to show how to differentiate authentic content from AI-generated content.

Critical thinking also plays a crucial role in identifying AI-generated content. Analytical skills enable students to examine texts in depth and detect subtleties that may indicate a non-human origin. To develop students' critical thinking skills you can use:

Questioning: Encourage students to always question and look for the 'why' behind the ideas presented in a text.
Text Analysis: Introduce text analysis activities to practice dissecting arguments and recognizing writing patterns.
Debate and Discussion: Use class debates to stimulate critical thinking and argumentation skills.

Text Comparison: Have students compare AI-generated texts with those written by humans, highlighting the differences.
Role-playing: Assign roles where some students write as if they were an AI, while others evaluate.
Individual reflection: Give students reflective exercises in which they analyze their own writing in relation to the criteria of authenticity.

Training in these aspects can prepare students to be competent evaluators, able to recognize AI-generated content and offer feedback that values authenticity and academic integrity

The benefits of peer review

Peer review is recognized for its multiple benefits in the educational context. Beyond its potential to detect AI-generated content, it also serves as an enriching pedagogical exercise that improves student engagement and learning outcomes.

Improving Engagement:

Shared responsibility: Peer assessment encourages students to become actively involved in the learning process, fostering a sense of shared responsibility.
Active participation: By critiquing each other, students participate more actively and are more involved in the course.

Learning results:

Deep reflection: The act of evaluating the work of others encourages deeper reflection on the subjects studied.
Skills acquisition: Students develop key skills such as critical analysis, argumentation and the ability to give and receive constructive feedback.

In an era where artificial intelligence (AI) is becoming increasingly integrated into our daily and academic lives, authenticity is becoming a precious currency, both for the content creator and the receiver. Peer review then emerges as an essential practice, a human filter that captures the essence of authenticity where machines might fail.

This form of peer assessment becomes a doubly beneficial exercise: it empowers students as critical assessors and authors aware of the originality of their work. They learn not only to identify the nuances that distinguish human creations from AI products, but also to value and refine their own ability to produce authentic, thoughtful work.

Moreover, peer assessment reinforces the collaborative aspect of learning, inviting students to see themselves not just as individuals on an academic journey, but as active members of a scholarly community where everyone contributes to the collective richness and integrity of knowledge.

The transition to authenticity-centric assessment practices underscores the importance of preparing students to navigate a world where AI is ubiquitous. By developing their critical judgment and honing their ability to evaluate the work of their peers, we equip them not only to detect AI-generated content, but also to value and preserve the undeniable signature of humanity in their intellectual creations.