CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation
By: Dazhen Deng , Sen Yang , Yuchen He and more
Potential Business Impact:
Teaches computers to understand and make charts.
Current chart-specific tasks, such as chart question answering, chart parsing, and chart generation, are typically studied in isolation, preventing models from learning the shared semantics that link chart generation and interpretation. We introduce CycleChart, a consistency-based learning framework for bidirectional chart understanding and generation. CycleChart adopts a schema-centric formulation as a common interface across tasks. We construct a consistent multi-task dataset, where each chart sample includes aligned annotations for schema prediction, data parsing, and question answering. To learn cross-directional chart semantics, CycleChart introduces a generate-parse consistency objective: the model generates a chart schema from a table and a textual query, then learns to recover the schema and data from the generated chart, enforcing semantic alignment across directions. CycleChart achieves strong results on chart generation, chart parsing, and chart question answering, demonstrating improved cross-task generalization and marking a step toward more general chart understanding models.
Similar Papers
Graph-Based Multimodal Contrastive Learning for Chart Question Answering
Computation and Language
Helps computers understand charts and answer questions.
ChartM$^3$: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension
CV and Pattern Recognition
Teaches computers to understand complex charts better.
Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering
Artificial Intelligence
Teaches computers to read charts better.