Does visualization help AI understand data?
By: Victoria R. Li, Johnathan Sun, Martin Wattenberg
Potential Business Impact:
AI understands data better with charts.
Charts and graphs help people analyze data, but can they also be useful to AI systems? To investigate this question, we perform a series of experiments with two commercial vision-language models: GPT 4.1 and Claude 3.5. Across three representative analysis tasks, the two systems describe synthetic datasets more precisely and accurately when raw data is accompanied by a scatterplot, especially as datasets grow in complexity. Comparison with two baselines -- providing a blank chart and a chart with mismatched data -- shows that the improved performance is due to the content of the charts. Our results are initial evidence that AI systems, like humans, can benefit from visualization.
Similar Papers
Understanding Why ChatGPT Outperforms Humans in Visualization Design Advice
Human-Computer Interaction
AI understands pictures better than people.
BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning
Computation and Language
Helps computers understand charts better.
Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks
Machine Learning (CS)
AI counts dots and finds outliers in charts.