https://www.arxiv.org/abs/2601.12146
Large Language Models (LLMs) are now central to code generation, but they often produce non-compiling or incorrect programs. We investigate how giving an LLM direct access to a real compiler (gcc) transforms it from a passive code writer into an active programming agent.
We conduct an extensive experiment on 699 real programming tasks in C, using models from 135 M to 70 B parameters. With compiler feedback integrated into the generation loop, the LLMs dramatically improve: compilation success jumps by 5.3 – 79.4 percentage points, syntax errors drop ~75 %, and undefined references drop ~87 %.
Interestingly, smaller LLMs with compiler feedback can outperform larger models without this access, suggesting that tools like compilers can compensate for model size and reduce energy/compute costs in software applications.
Overall, the study highlights the role software engineering tools play in practical LLM deployment, pushing us toward more interactive, feedback-driven code agents rather than one-shot generators. It’s a promising step toward combining NLP models with existing development ecosystems for better accuracy and efficiency.
