The state of AI-assisted data visualization research

Status: research snapshot, evidence cut 2026-08-14. Recheck by 2026-11-14, or earlier after a material model, harness, benchmark, registry, or analytics- assistant release.

Short version: executive summary. Forward view: What would unlock the next capabilities in AI-assisted data visualization? turns the moving-baseline evidence into dated, scoreable forecasts and keeps model-driven and technique-driven gains separate. Assessment view: How close is AI-assisted data visualization to the ideal? defines a context-sensitive ideal, scores nine current capabilities on one fixed maturity scale, reconstructs the same scorecard from 2017 through August 2026, and translates the forecasts into explicit score movements.

This report is for researchers, product teams, designers, and engineers who need to understand what AI-assisted visualization systems can currently do, which techniques appear to improve them, and how those claims are measured. It is a research synthesis and mechanism comparison, not a product ranking or buyer’s guide. Tools, agent skills, and commercial BI assistants appear as examples of techniques in practice; their documentation does not establish comparative quality.

Eight open-source repositories were inspected statically at the pinned revisions listed below. Eight core system papers were read in full, four initial benchmark and training studies were read for the moving-baseline analysis, three current commercial product surfaces were checked against first-party documentation, seven first-party model and harness release records were used for the shared timeline, and earlier papers in the underlying research packet were re-read where they controlled a comparison. Three established visualization frameworks were added to ground the distinction between task goals, narrative explanation, dashboards, and exploratory tools. A companion deep reading adds 18 agent-skill files or families, broader skills benchmarks, and the paired SciVisAgentSkills study; its report keeps registry installations, repository popularity, and behavioral evidence separate. A second specialized-vision report separates chart QA, parsing, OCR/layout, grounding, quality or integrity critique, and verifier/repair roles rather than treating every visual model as an interchangeable judge. A third companion, Learning data visualization when AI can make the chart, separates assisted performance from retained skill, distinguishes productive difficulty from removable friction, and forecasts how human practice should be reallocated as implementation becomes cheaper. A fourth companion, What AI-assisted data visualization benchmarks actually measure, translates benchmark names into their inputs, outputs, source distributions, graders, human context, supported claims, and nonclaims. No external repository, package, installer, skill, model gateway, or untrusted script was run. Repository tests and included evaluation results are observations about what the source contains, not independently reproduced results.

Start here: what agentic data visualization is trying to do

Agentic data visualization is not one task. A system may help a person discover a pattern, explain a finding, monitor a changing situation, make a decision, or produce and revise a durable visual artifact. These purposes can occur in a sequence—exploration may eventually become a published explanation—but they do not have the same success condition.

Rather than invent a new top-level taxonomy, this report borrows the discipline of Brehmer and Munzner’s established visualization-task typology: describe why the work is undertaken, what data and outputs it acts on, and how encoding and interaction support it. This matters acutely for agents. Returning a technically valid chart answers part of “how”; it does not show that the system served the intended human purpose.

These are overlapping goals, not product bins. Segel and Heer characterize narrative visualization by the balance between author-driven narrative flow and reader-driven discovery. Sarikaya and colleagues show that dashboards also vary by purpose, audience, interaction, and data semantics, and differ materially from exploratory visualization tools. A newsroom interactive may contain both guided explanation and exploration; a dashboard may monitor, analyze, teach, or communicate. The important step is to declare the sequence and apply the right evidence at each transition.

That produces a general requirement: every system should preserve the question, data, decisions, artifact, and evidence. A context contract should then select the appropriate authoring and evaluation method. A universal generator, critic, or acceptance score would erase differences the system needs to understand.

What is being compared

This landscape contains several kinds of thing that are easy to blur together. A paper can describe a tool, a repository can implement a paper, and a benchmark can be used to test several tools. They still make different kinds of claim.

The objects fit together roughly like this:

question + data
      |
      v
authoring system or product  <--- an agent skill can guide this work
      |
      v
structured plan, DSL, or code
      |
      v
renderer + browser  ---> chart or interactive dashboard
      |
      +--- deterministic checks test values and behavior
      +--- a benchmark supplies repeatable tasks and scoring
      +--- a human study tests how people author, understand, or decide

The eight core papers are not eight versions of the same product. Additional benchmark and training studies are introduced later where they answer the separate question of how the model baseline is changing.

Terms used throughout the report:

• Intermediate representation (IR): a structured, inspectable plan between a request and renderer code—for example fields, transforms, encodings, layout, and interactions stored as JSON-like data.
• Domain-specific language (DSL): a restricted language for one problem domain. It gives up some freedom so a compiler can reject or interpret output more reliably.
• Semantic model or semantic layer: a governed description of fields, measures, joins, permissions, and business meaning that sits above raw tables.
• Renderer: the software that turns a specification or code into pixels and interactive controls, such as Vega-Lite, ECharts, Matplotlib, or a BI product.
• Critic: a model pass that inspects code or a rendered artifact and proposes faults or repairs. A critic is another evidence channel, not an acceptance authority.
• Interaction trajectory: a recorded sequence such as open a tab, change a filter, and inspect the resulting state. A browser can replay it.
• Held-out task: an evaluation task not used as an example while building or prompting the system.
• Ablation: a controlled comparison that removes or changes one component to test whether that component actually caused the observed gain.

State of the field

This synthesis distinguishes repeated findings from environment-specific choices, direct negative or null evidence from merely unsupported claims, and demonstrated capability from open research questions.

Findings in one page

The field is not converging on a single agent architecture. It is converging on a more useful engineering pattern: reduce the part the model must improvise, externalize its decisions, and test the resulting artifact at the layer where failure can occur.

The strongest current techniques are:

1. A typed visualization or dashboard representation between intent and rendering. Raiven, NL2Dashboard, nvAgent, NL4DV-LLM, current Data Formulator/Flint, DashChat, and the best library-specific skills all constrain generation through a DSL, intermediate representation, or declarative spec. This improves syntax, edit locality, token efficiency, and the ability to validate individual decisions. It does not establish that the underlying question or takeaway is worthwhile.
2. Deterministic work for deterministic claims. Compilers, schema checks, aggregation checks, executable code, browser traversal, replayed interactions, and regression comparisons outperform asking a model to pronounce an artifact correct. The model remains useful for ambiguity, semantic mapping, critique, and repair; it should not substitute for arithmetic or runtime evidence.
3. Grounding in the actual data contract. Governed BI products increasingly bind natural language to semantic models, field descriptions, verified queries, permissions, sample values, and business glossaries. This is more consequential than assigning a generic agent an analyst persona.
4. Mixed-initiative authoring. Data Formulator 2 and DashChat make natural language one control surface among direct manipulation, structured selection, visible transformed tables, history, branching, and reversion. This has stronger human-use evidence than prompt-only generation.
5. Evaluation over the rendered artifact and intended use. DashArena’s new contribution is not another chart score. A system authors an interaction trajectory; a browser replays it; the judge sees task, screenshots, schema, and execution evidence. Human agreement improves materially when interaction evidence is present. This is the closest current answer to the earlier gap around communicative and analytical value, but it is not a measure of reader comprehension, learning, retention, or decision quality.

“Multi-agent” is therefore not a transferable technique by itself. It earns a place only when a role has a different information boundary or tool contract: schema access, transformation execution, visual inspection, browser interaction, or independent acceptance evidence. A planner, composer, and validator can help because they manipulate different representations and evidence, not because their labels simulate a team. Equal-budget single-agent results and several ablation findings still argue against agent count as a default quality lever.

Where the literature converges

The convergence is a common engineering posture, not a standard product architecture:

• Make intent inspectable. Independent systems repeatedly put a visible structure between the request and the render: an analytic specification, dashboard IR, DSL, semantic model, or chart contract. The vocabulary differs; the function is to expose fields, transformations, encodings, interactions, and edits for validation and correction.
• Use deterministic execution for deterministic claims. Recompute values, execute queries, compile specifications, inspect bindings, and replay browser actions. Models can map ambiguous language, criticize, and repair. They should not substitute for arithmetic or runtime evidence.
• Evaluate the delivered artifact. Returned code, structural similarity, and a nonblank render stop too early. Interactive work requires observable state and behavior; explanatory work requires final-context reading and a human gate when comprehension is claimed.
• Ground generation in the real data contract. Governed fields, measures, joins, permissions, verified queries, samples, units, and source vintage are more consequential than an “analyst” persona.
• Keep people able to steer and recover. The strongest authoring evidence favors visible transformed data, direct controls, small edits, history, branches, and reversion over prompt-only regeneration.

Where approaches legitimately diverge

Several competing approaches can each be right in different environments:

• Direct code versus a DSL and compiler. A restricted language can remove coding failures in a stable, specialized grammar; Raiven’s benefit is strongest in scientific visualization. Direct frontier models approach its reported performance in ordinary information visualization, where compiler overhead and an expressiveness ceiling may not pay for themselves.
• Prompt-first versus mixed initiative. Prompt-first creation is proportionate for a simple, reversible one-off. Exploratory or governed work benefits more from direct controls, visible state, history, and correction.
• Open-file tools versus governed BI. Portable code can work across renderers. A native BI assistant can instead inherit maintained semantic models, permissions, and worksheet state. Neither advantage transfers automatically to the other setting.
• One model versus specialized roles. A separate role can help when it owns different tools or independent evidence: schema access, execution, rendered inspection, or browser replay. Dividing one context into persona-labeled roles adds cost without establishing a new capability.
• Static inspection versus interaction trajectories. A static output may be adequately tested through data and final-render evidence. An interactive output needs action coverage and resulting state. A model-authored trajectory is still only a reproducible claim about intended use, not observed user behavior.
• Universal representation versus renderer-specific contracts. A universal IR promises portability; a narrow local contract preserves more expressiveness and is cheaper to maintain. The right boundary is an empirical question.

Direct negative, null, and conditional findings

• DashArena’s deterministic rules agreed with humans only 42.4% of the time, versus 79.8% for its full evidence judge. Removing interaction evidence reduced agreement by 8.1 percentage points.
• No DashArena model exceeded 86% render success or 74% interaction replay. Among 30 cases that executed cleanly but failed overall, 21 contained semantic defects.
• NL2Dashboard found diminishing returns from repeated critic rounds and recommends zero or one round, supporting capped repair rather than indefinite deliberation.
• Removing nvAgent’s processor slightly improved GPT-4o’s average result while harming weaker and multi-table settings. The component is conditionally useful, not universally beneficial.
• Raiven’s advantage was concentrated in scientific visualization; the DSL did not produce the same margin in ordinary information visualization.
• PlotGen’s human review marked 40.5% of outputs completely accurate and another 24.5% somewhat accurate. A sophisticated multimodal loop still produced many materially imperfect plots.
• DashChat users identified mock-data and domain mismatch, acronyms, direct-edit limitations, and styling limits. Rapid prototyping did not erase the need for domain context and correction.

Unsupported shortcuts are a different category

An unsupported claim is not proof of harm. It is simply insufficient evidence for acceptance. The AntV saved “success” results did not include browser render outcomes. The prompt-only packages in the initial sample have no behavioral with/without proof. LIDA’s code-and-text evaluator does not inspect the rendered chart. A validator that fails open is not an acceptance mechanism. Stars, installs, code returned, a nonblank render, synthetic personas, or one vision-model score may be useful inputs or baselines; none establishes data fidelity, integrity, reader understanding, or decision quality.

The model and harness baseline is moving

Shared timeline: September 2023 through August 2026

The three lanes below align general model and harness releases with the changing scope of visualization systems and benchmarks. They are a shared clock, not a causal model: release dates do not prove that a platform milestone caused a paper’s result, and publication dates lag the work. Scores remain comparable only inside the named studies.

The web presentation places every milestone on one proportional September 2023–August 2026 axis. Marker position encodes the date; label width does not encode duration. On narrow screens, the same evidence becomes a vertical chronology inside each explicitly named lane.

By the August 2026 evidence cut, the research question is no longer only whether a model can emit code for a plausible static chart. It is increasingly whether a model plus its harness can preserve meaning, construct a multi-view interactive artifact, exercise it in a browser, and supply evidence that it supports the analytical task. Reliability has not kept pace with that expanding scope.

The baseline has improved materially over the past 36 months, but the published record does not provide a clean visualization-specific learning curve from August 2023 to August 2026. Benchmarks changed, later papers often reran only a subset of models, judges changed, and current systems combine models with different prompts, tools, and test-time budgets. The defensible conclusion is a direction and a set of measured slices—not a universal annual improvement rate.

A fixed scorecard makes the remaining distance explicit

The companion capability scorecard applies one six-level rubric to nine dimensions: task framing; data, semantics, and provenance; construction; interpretation; integrity critique; steering and repair; interaction, responsiveness, and accessibility; reader or decision outcomes; and production, governance, and maintenance. The scale runs from “not demonstrated” through “outcome-proven.” It does not average the dimensions.

On that rubric, the August 2026 field is usable in bounded contexts for data grounding, ordinary construction, interpretation, and mixed-initiative repair; repeatable on bounded tests for integrity critique and delivered interaction; and only demonstrated—not established—for reader outcomes and the full production lifecycle. The same rubric can be applied retrospectively without pretending that unlike benchmark percentages share a common numerical axis. The historical reconstruction shows broad acceleration after 2023, but no common-core dimension yet crossing into field-wide deliverability. Context rows replace the generic ideal with the actual success condition for explanation, exploration, monitoring, decisions, production, scientific work, and reader assistance.

What the closest comparable slices show

The web presentation uses paired points for the before-and-after values below. Each measure keeps its own labeled scale and exact endpoints; readers may compare the two points inside a row, but not horizontal position across metrics. There is deliberately no combined score or common raw axis.

Plot2Code asks a multimodal model to reconstruct a scientific chart as code. An anonymous 2026 preprint re-reported direct, single-pass results on the same Python/Matplotlib subset and normalized scores over the full test set. Between the September 2023 GPT-4V result and the June 2025 Gemini 2.5 Pro result:

The large movement was in visual and textual fidelity, not basic code execution. This is a useful comparison, but it is one preprint’s reconstruction of one chart-to-code benchmark, not a field-wide time series.

CharTide, a peer-reviewed ACL 2026 training study, supplies a second same-paper comparison of GPT-4o and GPT-5 on three chart-to-code benchmarks. GPT-5 improved ChartMimic high-level similarity from 87.7 to 94.7, Plot2Code text match from 52.6 to 61.9, and ChartX’s five-point score from 2.61 to 3.59. Execution moved much less. The pattern is again uneven: generation is becoming more faithful, while harder visual reasoning and semantic alignment still leave substantial headroom. CharTide also shows that a small chart-specialized model can match or exceed a larger general model, so raw frontier scale is not the only route to improvement.

DashArena shows what the 2026 frontier can attempt that earlier chart benchmarks barely measured: a current general model can produce a multi-view interactive dashboard and a replayable intended-use trajectory from an open-ended task. Its top model was competitive with the anonymized human baseline in aggregate preference. Yet no model exceeded 86% render success or 74% replay success, and execution-clean semantic failures remained common. Breadth has expanded faster than reliability.

Dashboard2Code makes another part of dashboard behavior measurable. Its 180 Plotly Dash dashboard-code pairs cover 20 visualization types and eight callback patterns, with 450 interaction tasks. The best reported configuration scored 79.4 overall and 64.2 on the most complex interaction level. DOM access raised one model’s successful UI exploration from 20.61% to 64.58%, showing the value of exposing executable structure rather than relying on pixels alone. The benchmark also exposes a particularly dangerous failure: an interface can respond while hidden state or a transformation is factually wrong. It uses a fixed 1920×1080 viewport and excludes animation and popups, so mobile, responsive, and animated behavior remain outside the result.

Chartography asks a different question: can a model read difficult charts used in professional work? Its 100 practitioner-authored tasks span 12 domain labels and were independently verified by three experts per task. Thirty model configurations were run twenty times per task. The best tested configuration reached 45.0% mean pass@1. Greater reasoning effort helped in eleven of twelve paired comparisons, but the median gain was only 4.5 percentage points and longer reasoning often elaborated an initial visual misread. Because the tasks were deliberately screened for difficulty, 45% is not a prevalence estimate over all professional charts. It is strong evidence that success on basic visual-literacy or chart-QA sets does not imply dependable professional reading.

Scaffolding still helps, but generic instruction is already losing value

The most informative ablations do not say “more prompting is better.” They say that scaffolding helps when it adds a missing representation, tool, or evidence channel.

• In Text2Vis, GPT-4o’s direct pass rate was 26%. Three examples left it at 26%; retrieval plus three examples reached 31%. One structured answer-and-code feedback round reached 42%. Adding visual feedback improved the visual subscores but slightly lowered the final pass rate to 41%. Targeted feedback helped; examples alone were null; another feedback modality was not monotonically better.
• In the original Plot2Code study, adding detailed conditional requirements generally increased resemblance but reduced execution. Gemini Pro’s pass rate fell from 68.2% to 55.3%; the authors found no clear advantage for chain of thought or Plan-and-Solve over the default prompt.
• In nvAgent, removing the processor slightly improved GPT-4o on average while hurting weaker and multi-table cases. This is exactly what a moving baseline should produce: scaffolding that compensates for one generation of model can become redundant or obstructive for another.

The current operational baseline is also no longer a bare model call. Coding harnesses increasingly arrive with planning, repository search, execution, browser inspection, and packaged visualization guidance. A no-skill model is still a useful diagnostic control, but it is not the realistic alternative to a application-specific extension. The decision baseline must be the current model plus the normal harness, its default tools, and any ambient skills or instructions.

What can and cannot be projected

The observed direction supports three bounded forecasts:

1. Likely to commoditize: syntactically valid chart code, conventional chart selection, basic styling, routine repair, and generic “inspect your render” advice. These should be short, replaceable defaults, not a large permanent application-specific doctrine.
2. Likely to migrate into models or harnesses: generic planning, self-review, browser/tool use, and library navigation. A visualization system should consume these when they work and avoid duplicating their orchestration.
3. Unlikely to be solved by baseline capability alone: the local question, audience, semantic definitions, authoritative data, denominator, source custody, consequence, publication boundary, environment-specific evidence, and real-reader outcomes. These are facts and authorities the model does not acquire merely by becoming better at code or vision.

Linear projection would be false precision. The practical forecast is that the value of generic instruction will decay fastest, the value of executable local evidence will persist, and the value of context and authority boundaries will increase as agents become capable of taking more consequential action.

How evidence was graded

The same word—“works”—covers incompatible outcomes in this literature. This report keeps seven layers separate:

Passing a lower layer is a prerequisite, not a proxy for all higher layers. “Rendered,” “valid,” “high similarity,” and “preferred by a vision model” are different claims.

Evidence labels used below:

• A — comparative human/outcome evidence: a relevant controlled user or human-calibrated task study, with material caveats still reported.
• B — comparative benchmark evidence: a held-out or public task set with explicit metrics and baselines, but not a direct reader outcome.
• C — executable mechanism evidence: inspectable implementation, tests, or evaluation fixtures without persuasive independent efficacy evidence.
• D — product or author claim: first-party description, demo, popularity, or anecdote without a usable comparative outcome.

Technique map

Research and implementation landscape

This landscape shows where research techniques appear in working systems, agent-skill packages, and governed BI products. The entries are examples and evidence records, not product scores: a paper can support a bounded performance claim, a repository can expose an implementation, and product documentation can describe a contract, but those are different kinds of evidence.

Full tools and research systems

Skill and plugin packages

These packages make a useful maturity ladder:

static advice
  -> versioned constraints and on-demand retrieval
  -> explicit design/build artifacts
  -> deterministic source/code checks
  -> final renderer/browser inspection
  -> interaction coverage and evidence receipts
  -> held-out behavioral comparison with and without the package
  -> human outcome study

SciVisAgentSkills reaches the paired-ablation rung for specialized scientific work, but not independent reproduction or a human outcome. Vizro reaches furthest on execution evidence; AntV has the largest included retrieval/code benchmark; the OpenAI skill has the broadest chart-contract and delivery QA. Those are different strengths and should not be collapsed into an install count or one “best skill.” The companion skill-package deep dive compares 18 files or families and separates registry installations from repository popularity and behavioral evidence.

Governed commercial environments

The commercial systems are especially environment-dependent. Their main advantage is not a universally better model. It is custody of semantic models, permissions, field metadata, verified queries, authoring state, and the native renderer. That advantage does not transfer to an open-file editorial workflow unless another system is given equivalent data contracts.

The evaluation frontier

DashArena materially changes the map

DashArena, published 2026-08-11, contains 234 open-ended tasks derived from high-quality Tableau Public dashboards across 14 clusters. A candidate returns a single-file ECharts dashboard and a structured two-turn interaction trajectory. A Playwright executor replays the trajectory and gives the pairwise judge task context, screenshots, schema, and execution evidence.

The strongest results are about the evaluation method:

• Six dashboard-experienced human annotators labeled 100 pairs, three labels per pair; 99 were evaluable. Human inter-rater agreement was only moderate (Fleiss kappa .384), and 52 pairs were unanimous. Dashboard quality is not a naturally objective scalar.
• The distilled DashJudge-8B agreed with humans 79.8% of the time (kappa .600). Removing interaction evidence reduced agreement to 71.7% (kappa .441), while deterministic rules alone achieved 42.4% (kappa .095). Intended-use evidence added 8.1 percentage points.
• A 100-trajectory audit found 99.8% valid targets and 96% all-pages coverage, but per-page control coverage was 80.8%. Of 841 actions, 97.3% were attributable and 93.3% matched the intended behavior; the other 6.7% exposed errors rather than hiding them.
• On 120 held-out tasks, the top model, GPT-5.5, scored 1449 versus a human baseline at 1321, with overlapping uncertainty. The authors explicitly treat the human dashboard as a practical anchor, not an oracle, because it predates the derived task.
• No model exceeded 86% render success or 74% trajectory replay success. In a targeted 100-failure audit, construction/runtime caused 26.9%, interaction 25.4%, data computation/binding 22.4%, presentation/readability 16.4%, and analytical issues 9%. Of 30 execution-clean failures, 21 still had semantic defects.

This qualifies the earlier conclusion that communicative-value evaluation was empty. DashArena now provides a serious, human-calibrated measure of task-grounded analytical support and interaction quality. It does not reverse the conclusion about readers. The benchmark does not observe whether a target audience comprehends an argument, learns a concept, remembers the message, or makes a better real decision. Its pairwise aggregate is also not a universal taste or audience model. The right update is “a major middle layer now exists,” not “communicative value is solved.”

Dashboard2Code exposes state that a screenshot can hide

Dashboard2Code reconstructs interactive Plotly Dash applications from screenshots, optional DOM, and interaction. Its benchmark combines 58 real-world seed dashboards with 122 generated examples, then checks visual fidelity, code, dynamic browser behavior, and 450 interaction tasks. Ninety generated dashboards were also scored by three visualization- experienced graduate evaluators; the final automatic metric correlated .781 with their ratings.

The most important result is not one total score. DOM access sharply improves exploration, complex callbacks remain harder, and systems can produce a visually responsive but factually wrong state. A static image judge cannot see that failure. The scope boundary matters just as much: Plotly Dash only, fixed desktop viewport, no animation or popups, and no reader study.

Chartography makes professional reading a separate capability

Chartography, released 2026-08-11, contains 100 difficult professional chart-reading tasks across 12 domain labels. Each task was authored by a practitioner and independently verified by three experts, including an acceptable answer range. The best of 30 frontier configurations reached 45.0% mean pass@1 over twenty trials per task.

Higher reasoning effort usually helped, but not enough to erase the main failure: sparse axes, 3D projections, contours, and domain conventions can be misread before reasoning begins. More tokens then produce a longer explanation of the wrong visual premise. The benchmark is adversarially difficulty- screened, so it does not say models fail on 55% of ordinary charts. It does say that a general “chart literacy” score is too broad a release gate for professional, consequential reading.

Raiven narrows generation failure in a particular environment

Raiven’s .988 VMPC is strong counterevidence to treating direct-code chart generation failure rates as immutable. A formal DSL plus compiler can nearly eliminate many specified-mark, encoding, linking, hallucination, and execution failures. But the benchmark fully specifies the target visualization, and the largest delta occurs in 3D/scientific tasks where generic web code generation is weak. In ordinary 2D information visualization, direct frontier models nearly match Raiven on its own metric.

The transferable claim is conditional: when an environment has a stable visual grammar, recurring hard syntax, and deterministic rendering, invest in the representation/compiler. It is not evidence that a DSL can choose the right question, identify a misleading comparison, or replace final human judgment.

Benchmark construction is part of the technique

The companion benchmark crosswalk documents the input object, requested output, source distribution, grader, human context, supported claim, and nonclaim for the benchmarks carrying the main conclusions in this review. It reveals a measurement progression:

1. exact or near-exact code/spec matching;
2. syntactic legality and render success;
3. data/encoding correctness against a reference;
4. chart extraction, visual grounding, question answering, and comparison;
5. integrity or quality judgment with human calibration;
6. interaction state and task-grounded replay;
7. actual user performance, comprehension, or decision outcome.

Many impressive percentages in tool repositories live at levels 1–2. nvAgent and Raiven reach level 3 in bounded ways. ChartQAPro, Chartography, ChartDiff, and the two 2026 PolyChartQA benchmarks expand level 4 across realistic, professional, comparative, multi-chart, and multilingual cases. Misviz, VisJudge, and misleading-chart robustness studies measure parts of level 5. Dashboard2Code and DashArena reach level 6. Data Formulator 2 and DashChat provide small, contextual evidence at level 7 for authoring/prototyping experience, not for reader comprehension or decision quality. No one result spans the stack.

Three construction effects now deserve the same attention as model scores. Human-authored multi-chart questions were up to 27.4 percentage points harder than model-generated questions; ChartDiff’s lexical-overlap metrics favored systems that its human-aligned judge rated much worse; and misleading-chart system rankings changed between controlled synthetic and heterogeneous real charts. A benchmark is a task, distribution, and grader—not a neutral name.

Research gap ledger

An open question is not the same as an empty field. Some gaps now contain a bounded study, some contain evidence that a failure occurs, and some still lack the measurement needed to make a decision. The status language below is literal:

• Measured but bounded means a public comparison exists but does not yet generalize to ordinary production.
• Failure demonstrated means research establishes that the failure occurs, not its population frequency or best remedy.
• Partly answered means a human study covers one task, audience, or time window while leaving consequential transfer questions open.
• Open means the reviewed corpus does not contain direct outcome evidence for the question as posed.

This ledger changes the research agenda in two ways. First, it prevents a single new paper from being narrated as “the gap is solved”: the relevant row moves only as far as the evidence permits. Second, it makes research pursuits testable. A skill-package survey must eventually lead to an ablation, not a popularity ranking. A specialized-vision survey must identify a same-task, equal-budget critic comparison, not merely another chart-question-answering leaderboard.

Active refresh triggers

The following releases or studies would materially change one or more rows:

• DashArena releasing its task set, executor, and judge for independent reproduction, or adding coding-agent, non-Tableau, numeric-fidelity, and real user-outcome tracks;
• Raiven releasing code and third parties reproducing its scientific- visualization results with independent graders and exploratory tasks;
• a public equal-budget visualization study isolating role/tool decomposition from extra inference;
• independent current-model evaluations of generic, dashboard, accessibility, mobile, and explanatory-visualization skills against the normal harness;
• same-task, equal-budget routing comparisons among deterministic checks, current general vision, and narrow chart specialists on end-to-end critique;
• reader studies measuring comprehension, retention, trust calibration, accessibility, mobile use, or consequential decisions from generated visualizations;
• current Data Formulator studies covering governed sources, persistent agent threads, and free exploration rather than reproduction;
• commercial BI vendors publishing reproducible error rates or user outcomes stratified by semantic-model quality and task type; and
• evidence that small, local deterministic contracts fail to transfer across the renderer and publication environments a system actually serves.

Sources inspected

Pinned repositories

• Microsoft Data Formulator at 5d4f7b3, MIT.
• Microsoft LIDA at d892e20, MIT.
• HoloViz Lumen at 7d587f3, BSD-3-Clause.
• McKinsey Vizro at b1d7b11, Apache-2.0.
• AntV chart visualization skills at b47f2fe, MIT.
• Markdown Viewer skills at a3afd45; no repository license identified in the captured metadata.
• OpenAI role-specific plugins at fe5608d, MIT.
• NTCoding claude-skillz at 21c6101; no repository license identified in the captured metadata.

Papers read in full for this update

• Wang and Deng, DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation, 2026.
• Irger et al., Raiven: LLM-Based Visualization Authoring via Domain-Specific Language Mediation, 2026.
• Shi et al., NL2Dashboard: A Lightweight and Controllable Framework for Generating Dashboards with LLMs, 2026.
• Ouyang et al., nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow, ACL 2025.
• Shen et al., DashChat: Interactive Authoring of Industrial Dashboard Design Prototypes through Conversation with LLM-Powered Agents, 2025.
• Wang et al., Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way, CHI 2025.
• Sah et al., Generating Analytic Specifications for Data Visualization from Natural Language Queries using Large Language Models, 2024.
• Goswami et al., PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback, 2025.

Human-outcome studies added by the gap-ledger pass

• Kuo et al., Vibe Visualizing: Exploring the Human-AI Interaction Dynamics in AI-Assisted Visualization, 2026 preprint: 20 novices, 60 sessions, and 175 charts.
• Kwon et al., Visualizationary: Longitudinal Critique of Visualization Design, 2024 preprint: 13 designers working on self-selected visualizations over three to five days, plus three expert raters.
• Yan et al., The Effects of Generative AI Agents and Scaffolding on Visual Analytics Comprehension, 2024 preprint: randomized 117-person comparison of data stories, passive GenAI, and proactive scaffolded GenAI.
• Ardito et al., AI-Supported End-User Development for Data Visualization, 2025: one-company exploratory study with eight interviews and three design-probe sessions.

Benchmark and training studies for the moving-baseline analysis

• Wu et al., Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots, Findings of NAACL 2025.
• Rahman et al., Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text, EMNLP 2025.
• Zheng et al., CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution, ACL 2026.
• RRVF: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback, anonymous ICLR 2026 submission. Used only for its clearly labeled same-benchmark historical table and treated as preprint evidence.
• Li et al., Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards, 2026 preprint: 180 dashboards, 450 interaction tasks, and a 90-dashboard human validation of the evaluator.
• Chartography: A Benchmark for Professional Chart Understanding, 2026 preprint: 100 practitioner-authored, three-expert-verified tasks and 30 model configurations.
• Xu et al., POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering, ACL 2026: 22,606 charts and 26,151 QA pairs across ten languages.
• Ymyang et al., Chart-MRAG: Benchmarking Multimodal Retrieval Augmented Generation on Chart-based Documents, ACL 2026: 4,738 expert-validated QA pairs across chart-bearing real-world documents.
• Ye, ChartDiff: A Large-Scale Benchmark for Comprehending Pairs of Charts, ALVR 2026: 8,541 chart pairs and a direct lexical-versus-human-aligned metric comparison.
• Efat et al., Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts, Findings of ACL 2026: 534 multi-chart figures and separate human- and model-authored questions.
• Shu et al., FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models, ACL 2026: 1,200 real financial charts, 7,016 questions, and 26 evaluated models.
• Tonglet et al., Protecting multimodal large language models against misleading visualizations, ACL 2026: controlled vulnerability and six-intervention study over misleading and non-misleading charts.
• Laskar et al., Lost in Translation: Do LVLM Judges Generalize Across Languages?, Findings of ACL 2026: multilingual multimodal judge evaluation with a chart-centric subset.

Reader and accessibility study added by the open-question pass

• Touching or Chatting: LLMs and Tactile Charts for Blind and Low-Vision Chart Learning, 2026 preprint: 12 participants and 263 substantive queries; strong multimodal preference and spatial-model benefit without measured accuracy improvement.

Foundational task and environment frameworks

• Brehmer and Munzner, A Multi-Level Typology of Abstract Visualization Tasks, IEEE TVCG / InfoVis 2013.
• Segel and Heer, Narrative Visualization: Telling Stories with Data, IEEE TVCG / InfoVis 2010.
• Sarikaya et al., What Do We Talk About When We Talk About Dashboards?, IEEE TVCG / InfoVis 2018.

Current first-party product documentation

• Power BI: use Copilot with reports and semantic models, last updated 2026-05-26 when inspected.
• Tableau Agent FAQ, including July 2026 dashboard beta and current limitations.
• Looker Conversational Analytics overview, including LookML grounding, data agents, verified queries, and Advanced Analytics.

Model and harness milestone records

• GPT-4V system card, 2023-09-25.
• GPT-4o announcement, 2024-05-13.
• Claude 3.5 computer use beta, 2024-10-22.
• Claude Code research preview, 2025-02-24.
• Responses API and agent tools, 2025-03-11.
• GPT-5 for developers, 2025-08-07.
• Hosted computer environment and reusable agent skills, 2026.

Earlier evidence retained in the comparison

The underlying research packet also contains VisEval, MAST, Cleveland–McGill MLLM testing, Does It Run, VIS-Shepherd, Text2Vis, CoDA, RealChart2Code, equal-budget single-agent results, synthetic-persona studies, and LLM visualization literacy work. Those sources continue to control the claims about reader simulation, integrity versus readability, equal-budget coordination, and benchmark coverage. This update does not silently promote abstract-only captures from that packet to full-text evidence.

Update log

• 2026-08-14 — Initial public edition. Established the contextual goals typology, literature synthesis, technique history, benchmark timeline, convergence and divergence findings, demonstrated negatives, and research gap ledger.