TY - GEN
T1 - Behind Agentic Pull Requests
T2 - 23rd International Conference on Mining Software Repositories, MSR 2026
AU - Khelifi, Syrine
AU - Ouni, Ali
AU - Khemaja, Maha
N1 - Publisher Copyright:
© 2026 Copyright held by the owner/author(s).
PY - 2026/7/31
Y1 - 2026/7/31
N2 - AI coding agents are increasingly being adopted to autonomously author pull requests (PRs). While these agents can perform a wide range of tasks, little is known about how much human intervention is required to collaborate with them and integrate AI Agents contributions in practice. In this paper, we conduct an empirical study on human intervention in agent-authored PRs (APRs) as a measure of human effort and oversight. Using the AIDev dataset, we first compare how often humans intervene in APRs vs human-authored PRs (HPRs), as well as the resulting outcomes of this intervention. We then conduct a qualitative thematic analysis of human interventions in APRs and derive a taxonomy including 4 high-level categories and 42 intervention actions. Our results show that, human interventions occur less frequently in APRs than in HPRs (52.17% vs. 83.59%), but when it occurs in APRs, it requires higher review effort, including larger code churn and longer durations. Our taxonomy results show that most human effort is spent on guidance-level interventions with 58.02%, focusing on restricting the agent's actions and enforcing project-conventions, followed by decision-level interventions at 21.16%, direct code changes at 17.05% and operational-level intervention at 3.69%. Which indicates that, collaboration with coding agents, is shifting developer work from implementation to supervision, guidance and quality control.
AB - AI coding agents are increasingly being adopted to autonomously author pull requests (PRs). While these agents can perform a wide range of tasks, little is known about how much human intervention is required to collaborate with them and integrate AI Agents contributions in practice. In this paper, we conduct an empirical study on human intervention in agent-authored PRs (APRs) as a measure of human effort and oversight. Using the AIDev dataset, we first compare how often humans intervene in APRs vs human-authored PRs (HPRs), as well as the resulting outcomes of this intervention. We then conduct a qualitative thematic analysis of human interventions in APRs and derive a taxonomy including 4 high-level categories and 42 intervention actions. Our results show that, human interventions occur less frequently in APRs than in HPRs (52.17% vs. 83.59%), but when it occurs in APRs, it requires higher review effort, including larger code churn and longer durations. Our taxonomy results show that most human effort is spent on guidance-level interventions with 58.02%, focusing on restricting the agent's actions and enforcing project-conventions, followed by decision-level interventions at 21.16%, direct code changes at 17.05% and operational-level intervention at 3.69%. Which indicates that, collaboration with coding agents, is shifting developer work from implementation to supervision, guidance and quality control.
KW - Agentic AI
KW - Human Intervention
KW - Pull Request
UR - https://www.scopus.com/pages/publications/105047061111
U2 - 10.1145/3793302.3793586
DO - 10.1145/3793302.3793586
M3 - Contribution to conference proceedings
AN - SCOPUS:105047061111
T3 - Proceedings - 2026 IEEE/ACM 23rd International Conference on Mining Software Repositories, MSR 2026
SP - 842
EP - 846
BT - Proceedings - 2026 IEEE/ACM 23rd International Conference on Mining Software Repositories, MSR 2026
PB - Association for Computing Machinery, Inc
Y2 - 13 April 2026 through 14 April 2026
ER -