SIGCSE Virtual 2026
A Competency Framework for Assessing AI-Assisted Programming Work
Eli Young, Andy Luu, Michael Wehar
2nd ACM Virtual Global Computing Education Conference, November 12-15, 2026
Abstract
AI coding assistants have moved from autocomplete to agents that generate, run, and revise whole programs. As a result, software is increasingly being built by stating intent rather than writing code by hand. Furthermore, students are using these tools in their coursework. In an introductory course, 84 percent of students stated they used ChatGPT for their assignments, exposing a gap in assessment when these tools are at play. When an LLM writes the code, passing test suites alone does not show how well the student works with AI, because there are other expertise-revealing aspects of software development, like security. What separates a strong student (when AI is permitted) is how well they specify a task, direct an agent, and catch bad output. We present a competency framework that scores AI-assisted programming work along five competencies (problem solving, planning, prompting, verification, and agentic workflow) with an assessment design that offers three levels of AI availability. The framework runs in an online assessment platform we built and operate, and it reports interpretable results for each competency backed by evidence, though it is in early development and not yet widely used or validated. We describe the framework and its rationale while outlining a planned evaluation.
- computing education
- AI-assisted programming
- competency assessment
- student assessment
- AI coding assistants
Cite this paper
Eli Young, Andy Luu, and Michael Wehar. 2026. A Competency Framework for Assessing AI-Assisted Programming Work. In Proceedings of the 2nd ACM Virtual Global Computing Education Conference V.2 (SIGCSE Virtual 2026), November 12-15, 2026, Virtual Event, USA. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3795860.3846260
@inproceedings{young2026competency,
author = {Young, Eli and Luu, Andy and Wehar, Michael},
title = {A Competency Framework for Assessing AI-Assisted Programming Work},
booktitle = {Proceedings of the 2nd ACM Virtual Global Computing Education Conference V.2},
series = {SIGCSE Virtual 2026},
year = {2026},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/3795860.3846260}
}