A Theoretically Grounded Hybrid Ensemble for Reliable Detection of LLM-Generated Text
By: Sepyan Purnama Kristanto, Lutfi Hakim
Potential Business Impact:
Finds fake writing in schoolwork better.
The rapid proliferation of Large Language Models (LLMs) has blurred the line between human and machine authorship, creating practical risks for academic integrity and information reliability. Existing text detectors typically rely on a single methodological paradigm and suffer from poor generalization and high false positive rates (FPR), especially on high-stakes academic text. We propose a theoretically grounded hybrid ensemble that systematically fuses three complementary detection paradigms: (i) a RoBERTa-based transformer classifier for deep semantic feature extraction, (ii) a GPT-2-based probabilistic detector using perturbation-induced likelihood curvature, and (iii) a statistical linguistic feature analyzer capturing stylometric patterns. The core novelty lies in an optimized weighted voting framework, where ensemble weights are learned on the probability simplex to maximize F1-score rather than set heuristically. We provide a bias-variance analysis and empirically demonstrate low inter-model correlation (rho ~ 0.35-0.42), a key condition for variance reduction. Evaluated on a large-scale, multigenerator corpus of 30,000 documents, our system achieves 94.2% accuracy and an AUC of 0.978, with a 35% relative reduction in false positives on academic text. This yields a more reliable and ethically responsible detector for real-world deployment in education and other high-stakes domains.
Similar Papers
Weak Links in LinkedIn: Enhancing Fake Profile Detection in the Age of LLMs
Social and Information Networks
Makes fake online profiles easier to spot.
Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network
Computation and Language
Catches fake online reviews written by AI.
Assessing Classical Machine Learning and Transformer-based Approaches for Detecting AI-Generated Research Text
Computation and Language
Finds if writing is from a person or AI.