EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
EduGuard is a retrieval-augmented generation (RAG) tutoring framework designed for introductory programming education, integrating query understanding, instructor-approved retrieval, pedagogical strategy selection, rubric-aware generation, claim-level verification, and overreliance control. On the BILearn-CS benchmark, EduGuard achieved 90.1% correctness, 89.4% grounding, and 90.8% rubric alignment, with low rates of hallucination (4.9%) and direct-answer leakage (9.8%). In a small pilot study, it improved post-test accuracy from 68.4% to 81.2% and reduced overreliance compared to a GPT-4o-mini Tutor baseline. These results were obtained using a combination of Meta-Llama-3.1-8B-Instruct and DeBERTa-v3-large-MNLI models.
Why it matters: EduGuard demonstrates that safe and effective GenAI tutoring in programming requires explicit pedagogical controls, evidence verification, and deployment safeguards beyond standard retrieval or prompting methods.
Full story at: arXiv Computers and Society ↗