Larger Is Not Always Better: Leveraging Structured Code Diffs for Comment Inconsistency Detection
By: Phong Nguyen, Anh M. T. Bui, Phuong T. Nguyen
Ensuring semantic consistency between source code and its accompanying comments is crucial for program comprehension, effective debugging, and long-term maintainability. Comment inconsistency arises when developers modify code but neglect to update the corresponding comments, potentially misleading future maintainers and introducing errors. Recent approaches to code-comment inconsistency (CCI) detection leverage Large Language Models (LLMs) and rely on capturing the semantic relationship between code changes and outdated comments. However, they often ignore the structural complexity of code evolution, including historical change activities, and introduce privacy and resource challenges. In this paper, we propose a Just-In-Time CCI detection approach built upon the CodeT5+ backbone. Our method decomposes code changes into ordered sequences of modification activities such as replacing, deleting, and adding to more effectively capture the correlation between these changes and the corresponding outdated comments. Extensive experiments conducted on publicly available benchmark datasets-JITDATA and CCIBENCH--demonstrate that our proposed approach outperforms recent state-of-the-art models by up to 13.54% in F1-Score and achieves an improvement ranging from 4.18% to 10.94% over fine-tuned LLMs including DeepSeek-Coder, CodeLlama and Qwen2.5-Coder.
Similar Papers
Larger Is Not Always Better: Leveraging Structured Code Diffs for Comment Inconsistency Detection
Software Engineering
Finds code mistakes hidden in old notes.
CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
Software Engineering
Finds when code explanations don't match the code.
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
Software Engineering
Makes computer code easier to understand and write.