The Big Picture

If you’ve ever taught a large online course, you know the drill: online discussion boards become your go-to tool for student engagement. They’re scalable, easy to set up, and research tells us they work. But let’s be honest—they also come with their share of headaches: low participation, uneven contributions, or students going through the motions just to check a box.

So, is there a better or at least complementary way to drive deeper engagement and learning?

I explored this question with my colleague, David Joyner, in a recent large-scale study, comparing the effects of peer assessment and online discussion on online students’ learning. We analyzed data from 1,451 students in the same online graduate course over five years (Spring 2020 to Fall 2024). What we discovered has really shed light on how we think about strategies for student engagement.

What Did We Learn?

Peer assessment is a performance driver

Students who actively gave feedback to their peers—we called them “Collaborators” and “Reviewers”—significantly outperformed those who mainly stuck to discussion boards (“Discussants”) or barely participated at all (“Limited Contributors”). The takeaway? When students critically evaluate their peers’ work and provide structured feedback, their own learning takes a serious leap forward (Figure 1).

Online discussion still matters

But before we write off discussion boards entirely, here’s the plot twist: Our “Discussants” did significantly better than students with minimal engagement. Discussion boards aren’t the problem—they’re just not the whole solution.

Peer assessment was a much stronger predictor of both perceived peer support and overall course satisfaction

Students didn’t just perform better. They felt more supported and valued the course more highly when peer assessment was part of their experience (Figure 2).

Mean learning performance scores by student participation group

Figure 1: Mean learning performance scores by student participation group.

The figure breaks down student learning performance into four distinct levels of engagement. The data reveals a clear “step-up” effect:

  • Top Tier (Collaborators & Reviewers): Students who actively provided feedback to peers achieved the highest learning outcomes.

  • Middle Tier (Discussants): Students who primarily stuck to discussion boards performed well, but significantly lower than those who engaged in peer review.

  • Lower Tier (Limited Contributors): Students with minimal engagement in either activity saw the lowest performance scores.

peer assessment as a predictor

Figure 2: The Hidden Value of Peer Assessment. 

This figure measures student sentiment. Specifically, how much support they felt and how highly they rated the course.

We used a statistical measure called the “Standardized Coefficient (𝛽) to compare the impact of different activities. In plain English, this chart shows that peer assessment is a much stronger predictor of student satisfaction than discussion boards.

Even though discussion boards are often used to build community, the data suggests that the act of giving and receiving structured feedback actually does a better job of making students feel supported by their peers.

What Does This Mean for Teaching?

Online discussions are like the warm-up, and peer assessment is the workout.

When you combine them strategically, you are creating a learning rhythm

that helps students move from participation to transformation.

Think of it this way: online discussions are like the warm-up—they loosen things up, get students talking, and build comfort. Peer assessment is the workout—it stretches their thinking, builds intellectual stamina, and strengthens their ability to evaluate and reflect. You need both to build learning muscles that last.

When you combine them strategically, you’re creating a learning rhythm: online discussions offer easy entry points, while peer assessment provides structured, reflective engagement. Together, they help students move from participation to transformation.

But Here Are the Challenges…

Anyone who’s implemented peer assessment knows it’s not exactly plug-and-play. You’ve got to:

  • Design clear rubrics (which takes time to create and refine)
  • Train students to give quality feedback (not just “Great job!”)
  • Manage the logistics (who reviews whom? when? how many?)
  • Deal with fairness concerns (Is this grade reliable? Is there bias?)
These challenges are exactly why many instructors stick with online discussions despite peer assessment’s proven benefits. It’s just… a lot.

Enter AI: A Game-Changer?

AI-Augmented Peer AssessmentThis is where things get exciting (and a bit futuristic). What if AI could tackle those persistent pain points? Not replace the instructor or eliminate the human element, but handle the heavy lifting so we can focus on what matters most: student learning.

Based on my research and emerging work in this space, I’ve been developing a framework for designing AI-Augmented Peer Assessment. This framework includes seven elements organized across three stages of the peer assessment process. It aims at leveraging AI to streamline logistics, enhance quality feedback, ensure fairness, and ultimately, create a richer and more engaging learning experience for all learners.

The Framework: 7 Ways AI Could Transform Peer Assessment

7 Ways AI Could Transform Peer Assessment

Pre-Assessment: Before Students Start Reviewing

  1. AI-Assisted Rubric Design: Imagine typing in your learning objectives and having AI generate a solid first draft of your rubric—clear criteria, performance levels, the works. You review it, tweak it, maybe even involve students in refining it. What used to take hours now takes minutes.
  2. Intelligent Reviewer Assignment: Instead of randomly assigning reviewers (or spending an hour manually matching students), AI does the heavy lifting. It considers who has relevant expertise, balances workloads, avoids conflicts of interest, and even ensures students get diverse perspectives. Smart matching, zero headache.
  3. Personalized Feedback Coaching: Before students review their peers’ actual work, AI trains them. They practice on sample submissions, get immediate feedback on their comments (“This is too vague. Can you be more specific?”), and build their skills progressively. By the time they’re reviewing real work, they know what good feedback looks like.

During Assessment: While Students Are Reviewing

  1. AI Feedback Validation: As students write their peer reviews, AI acts like a real-time coach. “Your comment is constructive, but could you add a specific example?” or “You’ve addressed criterion #1 well, but what about criterion #2?” It catches issues before feedback gets submitted, ensuring quality and relevance.
  2. Automated Review Oversight: Here’s where AI becomes your teaching assistant. It monitors all reviews for quality, spots potential bias, flags suspiciously similar feedback (collusion, anyone?), and alerts you to cases that need your attention. You’re not drowning in data. Instead, you’re getting actionable insights about where to focus your limited time.

Post-Assessment: After Reviews Are Completed

  1. AI-Supported Feedback Synthesis: Students receive feedback from multiple peers—some helpful, some redundant, some contradictory. AI synthesizes it all into a coherent summary: “Here are the common themes, here’s where reviewers agreed, here’s where they diverged, and here are your top three priorities for revision.” Clear, actionable, not overwhelming.
  2. Analytics-Driven Interventions: AI analyzes patterns across all the peer assessment data. Which students are struggling? What common misconceptions keep appearing? Which rubric criteria are confusing? You get dashboards and insights that help you intervene early and refine your teaching. It’s evidence-based decision-making on steroids.

The Reality Check

Now, before you get too excited (or worried), let me be clear:

Some of these elements exist in various tools today. Others are still in the research phase. Building a comprehensive platform that does all seven seamlessly? That’s going to take serious development work, cross-disciplinary collaboration, and significant investment.

More importantly, we need to proceed thoughtfully. As we build these AI-powered solutions, we have to grapple with some hard questions:
  • How do we protect student privacy while using their data to improve learning?
  • How do we ensure AI doesn’t perpetuate or amplify bias?
  • How do we maintain transparency so students and instructors understand how AI is making decisions?
  • Where’s the line between helpful automation and over-reliance on algorithms?
The goal isn’t to replace instructors or eliminate human judgment. It’s to free us from the logistical nightmare so we can focus on what we do best: teaching, mentoring, and making thoughtful pedagogical decisions.
We’re not there yet. But with thoughtful development, ethical consideration, and continued research, I think we can get there.
What do you think? I’d love to hear your experiences with peer assessment: the good, the bad, and the “never again.”

Want to Dive Deeper?

If you are interested in the research behind this framework, here are some excellent resources:

  1. Darvishi, A., Khosravi, H., Sadiq, S., & Gašević, D. (2022). Incorporating AI and learning analytics to build trustworthy peer assessment systems. British Journal of Educational Technology, 53(4), 844–875.
  2. Ou, C. (2024). Engaging students with peer assessment. In C. Ou. Designing Socially Dynamic Digital Learning (pp. 98-112). Routledge. https://doi.org/10.4324/9781003368076
  3. Ou, C., & Joyner, D.A. (2025). Assess or discuss: Comparing peer assessment and online discussion for enhancing learning at scale.
    L@S ’25: Proceedings of the Twelfth ACM Conference on Learning @ Scale, Pages 222 – 226. https://doi.org/10.1145/3698205.3733927
  4. Ou, C., Thajchayapong, P., & Joyner, D.A. (2024). Open, collaborative, and AI-augmented peer assessment: Student participation, performance, and perceptions. L@S ’24: Proceedings of the Eleventh ACM Conference on Learning @ Scale, Pages 496 – 500. https://doi.org/10.1145/3657604.366470
  5. Sichterman, B., Noroozi, O., Boetje, J., van Ginkel, S., Khosravi, H., & Versendaal, J. (2025). Supporting peer learning with artificial intelligence: A systematic literature review. Innovations in Education and Teaching International, 62(5), 1648–1664.
  6. Topping, K. J., Gehringer, E., Khosravi, H., Gudipati, S., Jadhav, K., & Susarla, S. (2025). Enhancing peer assessment with artificial intelligence. International Journal of Educational Technology in Higher Education, 22(1).

How to Cite This Post?

 Ou, C. (2025, October 23). Designing AI-augmented peer assessment. https://engagedigitallearning.com/blog/designing-ai-augmented-peer-assessment/