How close is AI-assisted data visualization to the ideal?

Status: field-level capability scorecard, evidence cut 14 August 2026. Historical scores are retrospective synthesis, not contemporaneous benchmark totals. Forecast scores are conditional translations of previously published resolution tests, not promises.

Companions: The state of AI-assisted data visualization research, What the benchmarks actually measure, The practitioner and reader experience, and What would unlock the next capabilities?.

Executive answer

We can define an ideal state, but not as one universally best chart or one autonomous agent.

The common ideal is a system that can:

1. understand the actual human purpose and audience;
2. ground its work in authoritative data, definitions, and provenance;
3. construct a faithful and appropriate visual artifact;
4. interpret visualizations accurately, including difficult and unfamiliar ones;
5. detect misleading or uncertain evidence and calibrate its confidence;
6. support local, non-regressive revision;
7. work in the delivered interactive, responsive, and accessible surface;
8. improve the intended human outcome; and
9. remain governable, maintainable, and worthwhile over its lifecycle.

The context determines what the last mile means. An exploratory system should help an analyst form and test valid questions. A newsroom graphic should improve reader comprehension without weakening editorial integrity. An operational dashboard should support timely, correct response. A scientific visualization must preserve domain-specific meaning. There is no honest way to replace those different outcomes with one generic quality score.

On a six-level maturity scale, the August 2026 field is mostly at level 3, usable in bounded contexts, for data grounding, ordinary visual construction, chart interpretation, and mixed-initiative revision. It is mostly at level 2, repeatable on bounded tests, for integrity critique and delivered interaction. It remains at level 1, demonstrated but not established, for reader or decision improvement and for production efficiency, governance, and maintenance as one complete episode.

That is substantial progress, but it is not close to the ideal on the dimensions that determine whether a visualization should be trusted, shipped, or credited with helping a person. The frontier has broadened faster than outcome evidence has matured.

What “perfect” means

Perfect does not mean maximum autonomy. In consequential environments, a perfect system may refuse, expose uncertainty, request a definition, or preserve a human publication decision. The target is appropriate independence plus appropriate control, not removal of people from the process.

This definition borrows three established visualization frameworks rather than inventing a new theory of success:

• Munzner’s nested model separates domain problem and data, task and data abstraction, visual encoding and interaction, and algorithm. Its central warning is that an upstream error cascades: perfect rendering cannot rescue the wrong question or abstraction.
• Brehmer and Munzner’s task typology separates why the work is undertaken, what data and outputs it acts on, and how the visualization supports it.
• Lam and colleagues’ seven evaluation scenarios distinguish work-practice, analysis and reasoning, communication, collaboration, user performance, user experience, and algorithm evaluation. The evaluation question should determine the method; a convenient metric should not determine the claim.

Together they imply a fixed common spine and context-specific validation. The same artifact can be technically correct yet solve the wrong problem, impair a workflow, fail its reader, or become impossible to maintain.

The maturity scale

The score is the highest maturity supported by public evidence for a defined job and population. It is not a normalized benchmark percentage.

Level 5 is not required for every internal component. The common ideal is level 4 across the first seven and ninth dimensions, plus level 5 for the human outcome. Autonomy is a deployment choice, not a seventh maturity level.

The evidence confidence is reported separately:

• High: multiple direct studies or one strong benchmark with clear limits;
• Medium: bounded experiments, small studies, or converging mechanism evidence; and
• Low: demonstrations, product contracts, anecdotes, or a persistent absence that has not been tested directly.

No scores are averaged. A mean would let strength in code generation conceal a failure in integrity or reader outcome, and different contexts require different weights.

The nine dimensions

Current common-core scorecard

This is a scorecard of the strongest credible publicly demonstrated field maturity, not the median product or ordinary user’s experience. “3” means usable in at least a bounded declared context; it does not mean reliable across all use cases.

The shortest honest summary is therefore:

bounded creation and reading: usable
integrity and delivered interaction: repeatable
human outcome and operational afterlife: demonstrated, not established

What the ideal requires in different contexts

The common dimensions stay fixed. The release gates and outcome evidence change.

These rows are not product ratings. They show why the same global capability can be sufficient for one bounded draft and unacceptable for another environment.

How the scorecard was reconstructed through time

The retrospective series uses the same 2026 rubric at every date. Each score asks: what maturity had public evidence reached by that moment? It does not run current models on old benchmarks, award later knowledge retroactively, or pretend that different benchmark percentages share a common scale.

Dates were selected when the public evidence surface changed materially:

• 2017: declarative interactive grammars made structured construction and interaction programmable;
• 2019: learned visualization generation and formal design constraints made automated generation and recommendation explicit;
• 2020: natural-language queries produced inspectable analytic specifications;
• 2022: human-written, real-chart question answering made visual reasoning repeatable;
• 2023: LLM pipelines broadened goal, code, critique, and repair, while general multimodal models began reading renders;
• 2024: render–inspect–repair loops and mixed-initiative authoring added executable and human-steerable evidence;
• 2025: coding agents, database visualization, dashboard-specific systems, and targeted feedback expanded usable workflows; and
• August 2026: professional, multilingual, multi-chart, document, interactive state, replay, and judge-reliability benchmarks broadened the frontier.

Historical scorecards on the fixed scale

Three patterns matter more than any cell:

1. The representation substrate arrived early. Vega-Lite could compile concise interactive specifications in 2017. The missing capability was not rendering; it was reliable mapping from human purpose and real data into the specification.
2. Early learned generation was demonstration-grade. Data2Vis produced simple univariate and bivariate Vega-Lite charts, but reported failure conditions in roughly 15–20% of tests, phantom fields, and only a qualitative comparison. Draco made design knowledge testable as constraints, a durable mechanism that did not by itself understand purpose or outcome.
3. The scorecard moved when evidence crossed a maturity boundary. NL4DV showed structured natural-language specification in 2020 but explicitly lacked a large labeled benchmark. ChartQA made real-chart reasoning repeatable in 2022. LIDA made grammar-agnostic LLM generation repeatable on 57 datasets, but its quality evaluator read code rather than the rendered chart and was not calibrated to human outcomes.

The apparent plateau from 2025 to 2026 is not a claim that models stopped improving. The 2026 frontier expanded sideways into much harder objects and contexts. An ordinal maturity score changes only when evidence crosses a threshold; large within-level gains and harder new benchmarks belong in the supporting evidence, not in invented decimals.

Forecasts translated into score movements

The existing forecasts resolve against explicit future evidence. Translating them into this scorecard clarifies what each success would—and would not—change.

If all four 2027 tests pass, the visible frontier would change from “usable creation, repeatable critique and interaction” to “deliverable static creation, usable critique, and usable bounded interaction.” It would still leave the two largest gaps—human outcomes and operational afterlife—essentially untouched.

The forecast events are correlated, so their probabilities should not be added or multiplied into one expected score. The scorecard should move only when the named resolution evidence exists.

How to maintain the scorecard

Every future update should preserve five records:

1. the fixed maturity rubric and ideal target;
2. the exact population, task, environment, model, harness, and tools;
3. the evidence that justifies a threshold crossing;
4. the contexts to which the score does not transfer; and
5. the prior scorecard as a dated historical snapshot.

A benchmark improvement may change a score, confidence, both, or neither. A larger test that confirms the same maturity should raise confidence without inventing a higher level. A new hard distribution may reveal a narrower context boundary without implying that all earlier capabilities regressed. A field study may raise outcomes while leaving underlying model scores unchanged.

The most valuable missing evidence is not another aggregate chart leaderboard. It is a linked set of complete episodes: purpose and audience, authoritative data, accepted artifact, delivered interaction, representative human outcome, total effort, later correction, and ownership transfer.

Source map

The links in the score rows above lead to the evidence closest to each judgment. The groups below explain how those sources enter the scorecard. A source supports only the named claim; its presence is not an endorsement of a larger product, model, or field-wide conclusion.

The comprehensive benchmark crosswalk records each benchmark’s input, output, data source, grader, human context, supported claim, and important nonclaim. The forecast report owns the probabilities, confidence judgments, resolution tests, and annulment conditions translated into score movements above.

Judgment note

The scores are editorial synthesis constrained by those sources. They are not the output of a validated psychometric instrument, and they should not be used to rank products or models. Their purpose is to make the definition of progress, the remaining distance, and the evidence required to move explicit.

Update log

• 2026-08-14 — Initial public edition. Defined the ideal-state dimensions, fixed maturity rubric, August 2026 field assessment, historical reconstruction, contextual endpoints, and forecast score movements.