What the benchmarks actually measure
The same system can pass code checks, misread the finished chart, miss a deceptive axis, or fail after a dashboard control changes state. Read the evidence as cumulative obligations. No captured evaluation spans the whole episode.
Requested code, fields, marks, layout, or specification
Compile, render, load, and callback completion
Values, transformations, bindings, and encodings
Extraction, grounding, questions, and comparison
Deception, defects, and calibrated quality judgment
Control behavior, hidden state, and intended-use replay
Accepted work, comprehension, decisions, and access
Turn an image, source data, database question, or specified view into chart code, a visualization query, or a compiled artifact.
Plot2Code · VisEval · Text2Vis · RealChart2Code · Raiven
Execution and resemblance do not establish that the question, source, semantics, or design is appropriate for a person.
Extract values, locate marks, answer questions, detect deceptive encodings, or predict an expert-defined quality rating.
ChartQAPro · CHART-6 · Misviz · VisJudge · FinChart-Bench · Chartography
Chart reading is not chart authoring. A visual critic usually cannot verify raw data, audience fit, interaction, or reader outcome.
Compare chart pairs, localize and reason across multi-panel figures, retrieve charts and text from documents, or work outside English.
ChartDiff · multi-chart PolyChartQA · Chart-MRAG · multilingual POLYCHARTQA
Translated, generated, or carefully curated sources do not represent ordinary work across languages, institutions, devices, or local chart conventions.
Navigate, reconstruct, or generate an interactive dashboard; replay intended use; or observe a creator or reader performing a task.
DashboardQA · Dashboard2Code · DashArena · bounded authoring and reader studies
No evaluation follows one artifact through authoritative data, acceptance, delivery, mobile and assistive use, reader outcome, later correction, and maintenance.
Multi-chart accuracy fell on human-authored questions compared with model-generated questions.
ChartDiff specialists and pipelines scored higher on overlap while general models led on calibrated quality.
Table-based QA resisted misleading charts but sometimes damaged accuracy on non-misleading charts.