AI Efficacy Research

Why Do We Need AI Efficacy Research in Education?

According to surveys performed by the Digital Education Council (2024) and ACT (2024), 86% of college students and 46% of pupils in grades 10-12 now report using AI tools for their coursework.

As AI becomes ubiquitous for students, the role of the educator must now include helping students develop the skills to navigate AI thoughtfully, responsibly, and ethically, while still guiding them in their learning and coursework.

What is AI?

At its core, we can define AI as:

Technology that enables machines to perform tasks traditionally requiring human intelligence.

AI’s core capabilities—understanding, prediction, creation, optimization—are now being adapted for student learning and instructional support, bringing immense potential for student success.

Common types of AI technologies used in learning environments:

 

  • Machine Learning (ML): Algorithms that learn from data to make predictions or decisions. For example, identifying patterns in student submissions to flag off-topic work.
  • Deep Learning (DL): A powerful subset of ML, modeled on the human brain’s neural networks. It’s often used for image or speech recognition—but in learning, it powers more advanced feedback engines.
  • Expert Systems: These rule-based systems apply human-designed logic to reach decisions. For example, helping an instructor determine which students need additional support based on assignment scores.
  • Generative AI: These models create  new content—such as text, images, or even quiz questions—based on the patterns they’ve learned from large datasets. Tools like ChatGPT are part of this category.

Understanding how these AI technologies interact—and their limits—is essential for instructors and administrators alike in helping students succeed.

The Importance of Explainable Feedback

Artificial intelligence systems that are designed to make their decisions transparent, understandable, and interpretable by humans are referred to as Explainable AI (XAI). In educational settings, this is especially critical. When educators use AI to assess student work, guide feedback, or inform instruction, it is crucial for students to know why and how a decision was made in order to learn from this feedback.

According to Mersha et al. (2025), explainability is essential for “transparency, accountability, and trust” in AI applications. When educators and students can’t understand or question an AI-generated recommendation or score, the result is a loss of confidence, and in some cases, the erosion of fairness.

That’s why explainability isn’t optional in learning environments. It must be a design principle—not an afterthought.

Here’s what that looks like in practice:

 

  • Students can trace feedback to specific areas of their work and understand how to improve.
  • Instructors retain the authority to override AI-generated suggestions or grades.
  • Institutions can audit AI decisions and verify alignment with equity and academic policy.

In short, XAI ensures that AI in education functions not as a black box—but as a transparent tool that builds trust, supports equity, and reinforces instructional integrity.

Key Questions for Assessing An Effective AI Education Tool

  • Is the system explainable?
  • Can students understand how it works?
  • Are instructors able to review and override it?
  • What are the risks of bias or error?

The Undeniable Importance of Human Oversight

When AI recommendations have to potential to significantly affect students’ learning, having human oversight to verify those recommendations, with clear accountability mechanisms, become extra important. We believe students have the right to challenge AI-driven decisions and receive transparent explanations, aligning directly with OECD’s accountability principle.

Our Instructional AI is rooted in decades of educational research on how students learn best. Rather than layering AI on top of outdated classroom practices, we designed Packback’s platform to amplify what already works: inquiry, feedback, and metacognitive reflection.

Here’s how Packback’s approach reflects our educational values:

 

  • Students receive feedback they can understand and act on.
  • Educators remain in full control of grading and instructional choices.
  • Every AI-supported decision can be reviewed, explained, and refined.

Inquiry-Driven Learning: Sparking Curiosity and Engagement

Inquiry-based approaches foster critical thinking, collaboration, and deeper engagement. Students move from passive consumers of information to active participants in meaning-making.

Research Insight: The Community of Inquiry (CoI) framework defines effective learning as the interplay of social presence, teaching presence, and cognitive presence (Garrison, Anderson, & Archer, 2000).

Packback operationalizes this model by encouraging students to lead discussions through thoughtful, open-ended questions.

Students ask their own discussion questions, rather than responding to prompts. AI feedback helps them refine question clarity and depth in real-time. Built-in features and peer voting reinforce the social and teaching presence.

Formative Feedback and Iteration: Driving Mastery Through Action

Formative feedback—timely, specific, and actionable—has one of the strongest positive effects on learning outcomes (Hattie & Timperley, 2007). Rather than waiting until an assignment is graded, Packback delivers this kind of feedback while students write, providing guidance in the moment when it’s most useful.

Students improve faster when they’re given space to revise and reflect. When feedback is non-punitive and focused on the process rather than the final performance, it promotes a growth mindset and encourages students to take ownership of their learning. This supports the development of self-regulation and long-term academic success.

The Research Behind It:  

A landmark study by Black and Wiliam (1998) revealed the transformative power of formative feedback across subjects and grade levels. Their research showed that when students receive timely, constructive input, they engage more deeply with the material, regulate their own learning more effectively, and see significant academic gains—especially students who are lower-performing. This foundational work emphasizes that feedback should guide students toward mastery, not simply audit their current level of understanding. 

Packback’s real-time, AI-powered feedback loops are built with these principles in mind. Rather than offering feedback after a task is complete, Packback supports students during the writing process—encouraging reflection, revision, and deeper thinking before submission.

 

  • AI-powered Instant Feedback gives students suggestions on structure, clarity, and credibility as they draft their work.
  • Feedback emphasizes curiosity, communication, and critical thinking—not just correctness.
  • Students are encouraged to revise their work before submitting, creating feedback loops that support mastery-based learning.

By aligning with decades of research and focusing on the how of learning—not just the what—Packback’s feedback engine helps students grow into more thoughtful, confident writers and learners.

Metacognition and Self-Regulation: Helping Students Learn How to Learn

Metacognition—the ability to reflect on and regulate one’s learning—is a key predictor of academic success (Zohar & Barzilai, 2013). Packback supports metacognitive development by making students more aware of their learning process through iterative feedback and reflective writing.

Students who can monitor their own thinking are better equipped to set goals, persist through challenges, and apply what they’ve learned in new contexts.

 

  • Feedback is framed as a tool for reflection, not evaluation.
  • Students can iterate on their thinking through revision and question design.
  • Writing tutors and coaching tools guide students to ask better questions and assess their own reasoning.

Foundational Pedagogy: Mastery Learning

Mastery Learning, developed by Bloom(1968), ensures students achieve proficiency through structured feedback and iterative learning. Research confirms its effectiveness in improving retention, comprehension, and academic success. The core principles of mastery learning are:

 

  • Clear Objectives: Defined learning goals provide a roadmap for success.
  • Formative Assessment & Feedback: Continuous feedback helps students refine their understanding before final evaluations.
  • Corrective Instruction: Targeted interventions guide students toward mastery.
  • Individual Pacing: Students advance at their own pace upon demonstrating proficiency.
  • Enrichment for Mastery: Advanced learners deepen their knowledge beyond core requirements.

AI and Mastery Learning:

Packback’s AI-driven platform aligns with mastery learning by providing:

 

  • Real-Time, Formative Feedback: Instant, constructive insights enhance learning.
  • Personalized Learning Paths: Adaptive AI tailors instruction to individual needs.
  • Bias-Aware Assessment: Ethical AI ensures fairness in feedback and grading.

Mastery Learning, combined with AI, fosters equitable, effective education by enabling personalized, feedback-rich learning experiences that drive student success.

The Bottom Line: Pedagogy-First AI Works

When AI is built to support—not replace—evidence-based instructional practices, the impact is powerful. Packback’s integration of inquiry, feedback, and metacognitive scaffolding helps students develop skills that go far beyond a single assignment: critical thinking, curiosity, and the confidence to grow.

Our tools are not magic—they’re purpose-built, research-aligned supports that help students write better, think deeper, and take ownership of their learning. Educators retain control; students stay empowered.