Measuring Iterative Temporal Reasoning with TimePuzzles
By: Zhengxiang Wang, Zeyu Dong
We introduce TimePuzzles, a constraint-based date inference task for evaluating iterative temporal reasoning. Each puzzle combines factual temporal anchors with (cross-cultural) calendar relations, admits one or multiple valid solution dates, and is algorithmically generated for controlled, dynamic, and continual evaluation. Across 13 diverse LLMs, TimePuzzles well distinguishes their iterative temporal reasoning capabilities and remains challenging without tools: GPT-5 reaches only 49.3% accuracy and all other models stay below 31%, despite the dataset's simplicity. Web search consistently yields substantial gains and using code interpreter shows mixed effects, but all models perform much better when constraints are rewritten with explicit dates, revealing a gap in reliable tool use. Overall, TimePuzzles presents a simple, cost-effective diagnostic for tool-augmented iterative temporal reasoning.
Similar Papers
TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios
Artificial Intelligence
Helps computers understand time and events better.
SPAN: Benchmarking and Improving Cross-Calendar Temporal Reasoning of Large Language Models
Artificial Intelligence
Helps computers understand different calendars and dates.
TIME: Temporally Intelligent Meta-reasoning Engine for Context Triggered Explicit Reasoning
Machine Learning (CS)
Makes AI think smarter, faster, and more naturally.