To Tab or Not To Tab Accepted at ITiCSE 2026
Later that summer at ITiCSE 2026, the paper won the conference Best Paper Award. I wrote a follow-up post about the award, Jess' presentation, and the team behind the project.
I am excited to share that our paper, To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks, has been accepted for publication at ITiCSE 2026.
We have been asking a simple but important question: when novice programmers accept AI code suggestions, are they thoughtfully engaging with those suggestions or simply tabbing through them? Our paper introduces attention checks, deliberately incorrect AI suggestions that conflict with a programmer's immediate goals, as a way to study whether users notice, question, and critically evaluate AI-generated code before accepting it.
This Clover project matters to me not only because of the paper itself, but because it grew through collaborative undergraduate research and helped student researcher Jess Hutchison reach her first publication. I wanted this post to capture the work behind the paper: what we were studying, why it matters, and how the project became a place for students to grow into real research ownership.
Why This Work Matters
One of the biggest challenges in evaluating AI code completion tools is that we still lack reliable behavioral metrics for reflective use. It is easy to measure whether someone accepted a suggestion. It is much harder to measure whether they were genuinely paying attention, thinking critically, or noticing when the AI was wrong.
Our paper contributes two pieces that we hope will be useful to researchers and educators working in this space.
Attention Checks for AI Code Completion
We introduce attention checks as a method for studying critical engagement. These are intentionally incorrect suggestions that create a measurable moment of reflection. When a suggestion clearly conflicts with a programmer's current task, their response can reveal whether they are carefully evaluating the AI or passively accepting what appears in front of them.
A Taxonomy of Behavioral Metrics
The paper also develops a taxonomy of behavioral metrics for studying how programmers interact with AI code completion tools. That taxonomy became a foundation for the project and provides a new way to reason about reflective AI use in programming environments.
The Collaboration Behind the Paper
The Clover extension, study, was developed primarly by undergraduate students. Clover was a labor of love. It went through building and prototyping through my capstone course to production. Two aboslute all stars were Nick Ruckinski and Jamie Nguyen who built the Clover extension and its underlying architecture. They were a powerhouse duo, building a full clone of GitHub Copilot, then working relentlesly on our education line-by-line approach that
One of my favorite behind-the-scenes memories from the final push was a late-night paper session fueled by Chinese skewers from Yi Pin. My partner, Xuelin Wang, brought them to support us while we were working through the last stretch of the paper. Pictured here are my advisor, Dr. Stephen MacNeil, Kush Patel, Antonio Lazaro, Kenny Angelikas, and Xuelin. It is a small moment, but it captures a lot of what made this project meaningful to me: shared effort, generosity, and the kind of tired, chaotic collaboration that makes research feel deeply human.
Student Growth Behind the Work
What makes this acceptance especially meaningful to me is the student growth behind it. Jess Hutchison contributed to this work as an undergraduate researcher, and this paper became her first publication. Watching her grow through the project, gain confidence, and deepen her identity as a researcher has been one of the most rewarding parts of the experience.
This project also became part of Jess's path toward pursuing a PhD in the HCI Lab. That is one of the main reasons this publication feels important to me. It is not only a paper acceptance. It is a reminder that undergraduate research can help students grow into leaders, future scholars, and more confident collaborators in ways that last well beyond a single project.
Looking Back From ITiCSE
I am proud of what this paper contributes to conversations around AI, software engineering education, and computing practice. I am just as proud of what the collaboration represents: undergraduate research as a place where students can build ambitious things, learn how to think carefully, and discover larger futures for themselves.