Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning
By: Linze Chen , Yufan Cai , Zhe Hou and more
Potential Business Impact:
Makes AI judge cases fairly and explain why.
The rationality of law manifests in two forms: substantive rationality, which concerns the fairness or moral desirability of outcomes, and formal rationality, which requires legal decisions to follow explicitly stated, general, and logically coherent rules. Existing LLM-based systems excel at surface-level text analysis but lack the guarantees required for principled jurisprudence. We introduce L4M, a novel framework that combines adversarial LLM agents with SMT-solver-backed proofs to unite the interpretive flexibility of natural language with the rigor of symbolic verification. The pipeline consists of three phases: (1) Statute Formalization, where domain-specific prompts convert legal provisions into logical formulae; (2) Dual Fact and Statute Extraction, in which prosecutor- and defense-aligned LLMs independently map case narratives to fact tuples and statutes, ensuring role isolation; and (3) Solver-Centric Adjudication, where an autoformalizer compiles both parties' arguments into logic constraints, and unsat cores trigger iterative self-critique until a satisfiable formula is achieved, which is then verbalized by a Judge-LLM into a transparent verdict and optimized sentence. Experimental results on public benchmarks show that our system surpasses advanced LLMs including GPT-o4-mini, DeepSeek-V3, and Claude 4 as well as state-of-the-art Legal AI baselines, while providing rigorous and explainable symbolic justifications.
Similar Papers
Judicial Requirements for Generative AI in Legal Reasoning
Artificial Intelligence
Helps AI understand and argue legal cases.
NomicLaw: Emergent Trust and Strategic Argumentation in LLMs During Collaborative Law-Making
Artificial Intelligence
AI agents learn to make laws and persuade each other.
Thinking Machines: Mathematical Reasoning in the Age of LLMs
Artificial Intelligence
Helps computers prove math ideas like a scientist.