ExecExam
Introduction
Software engineering is not just about writing code; it’s about solving real-world problems with technology. One such challenge is providing scalable and effective feedback for programming students. ExecExam is a tool that tackles this issue head-on, automating Python assessments and enhancing learning experiences. In this post, I’ll discuss my contributions to the project, particularly with LiteLLM, and how ExecExam is now used at Allegheny College to support education.
What is ExecExam?
ExecExam is an automated assessment tool for Python programming tasks. It integrates with Pytest to test student code, providing detailed feedback on errors and improvements. More than just a testing framework, ExecExam offers enhanced educational support through large language models (LLMs), which analyze failing test cases and offer step-by-step suggestions for fixes.
My Contributions to ExecExam
My primary contribution to ExecExam was the integration of LiteLLM, a lightweight API for interacting with various LLM providers. By incorporating LiteLLM, we made it possible for students to receive context-aware, AI-driven feedback directly in their assessments.
This feature ensures that instead of just showing test failures, ExecExam provides tailored suggestions on why the failure happened and how to fix it. This significantly enhances the learning experience by encouraging students to think critically about their mistakes rather than just trial-and-error debugging.
Impact on Education
ExecExam is currently used at Allegheny College to assist both students and educators. With the LiteLLM integration, students can better understand their mistakes, and instructors can track common problem areas across their classes. The tool streamlines grading while fostering a more interactive and personalized learning environment.
By reducing the time spent manually reviewing code, instructors can focus on mentoring students rather than debugging their submissions. Meanwhile, students benefit from instant, constructive feedback, which enhances their learning and problem-solving skills.
PyCon 2025 Poster Session
I’m thrilled to share that ExecExam has been selected for a poster session at PyCon 2025! Our team will present how ExecExam enhances Python education through automation and AI-driven feedback. This opportunity allows us to showcase our work to a broader audience, discuss improvements, and explore future integrations. If you're attending PyCon, be sure to stop by our session!
Conclusion
Contributing to ExecExam has been an exciting journey in software engineering and educational technology. Seeing the tool adopted for real-world use at Allegheny College is incredibly rewarding. As we continue refining ExecExam, I hope it will serve as an example of how AI can be leveraged to improve education and software development.
If you're interested in the project, check out the source code on GitHub!