Human objective
Present, discover, compare, monitor, decide, learn, or produce.
The state of AI-assisted data visualization research.
As of August 2026: what capabilities current research studies, which techniques it tests, how those capabilities are measured, where results converge or conflict, and what remains unknown.
Main finding
The most reliable systems expose consequential choices and test the finished artifact. Generic guidance should remain only when it improves on the current model, tools, and built-in workflow.
For researchers and buildersUse this review to interpret the literature, design evaluations, and choose which mechanisms deserve local experiments.
Not a product rankingTools, skills, and BI products appear as examples of techniques in practice. Their documentation does not establish comparative quality.
What agentic visualization is for
Begin with the human purpose, not the chart or the agent.
Agentic data visualization is not one task. A system may help someone discover a pattern, explain a finding, monitor a changing situation, make a decision, or produce and revise a visual artifact. Those goals impose different requirements on autonomy, interaction, evidence, and human control.
An established backbone
Describe the goal, the data, and the method separately.
Brehmer and Munzner’s widely used task typology describes visualization work by why it is undertaken, what data and outputs it acts on, and how encoding and interaction support it. That distinction is especially important for agents: generating a technically valid chart answers “how,” but may still miss the intended human purpose.
Brehmer & Munzner typology ↗Data and output
Which entities, measures, relationships, uncertainty, state, and artifact are in scope?
Means
Encode, select, filter, navigate, derive, annotate, record, compile, or replay.
Human purpose
What the system must help with
Optimize for
Human control
Explain and present
Support a specific account of the evidence.
Journalism, public explanation, and narrative graphics usually have an authorial claim, sequence, and audience. The visualization may offer exploration, but it also has to fit the surrounding story and help a reader understand what the evidence supports.
- Optimize for
- comprehension · integrity · narrative fit · accessibility
- Human control
- high: claim, context, emphasis, and publication
Explore and discover
Help a person form and test questions.
Exploratory analysis begins without one fixed takeaway. The useful system exposes transformed data, alternative mappings, branches, history, and direct controls so the analyst can notice patterns, challenge them, and change direction.
- Optimize for
- iteration · ambiguity · correction · provenance
- Human control
- continuous: steer, inspect, branch, and stop
Monitor and respond
Maintain awareness of changing conditions.
Operational dashboards emphasize current state, thresholds, anomalies, and timely response. Sarikaya and colleagues show why dashboards deserve their own design space: at-a-glance reading, coordinated views, updating data, and shared awareness are not the same problem as free exploration.
- Optimize for
- recency · stable measures · alerts · action state
- Human control
- governed: thresholds, escalation, and response
Compare and decide
Relate evidence to alternatives, targets, or commitments.
Strategic and analytical BI needs trustworthy definitions, fair comparisons, permissions, and enough context to understand tradeoffs. The objective is not merely to surface an interesting pattern but to support a consequential judgment.
- Optimize for
- semantic custody · comparisons · uncertainty · auditability
- Human control
- high where decisions are consequential
Produce, revise, and reuse
Create a durable visual artifact or application.
Authoring tools, coding agents, and scientific systems are judged partly by execution, editability, reproducibility, and delivery. Scientific and 3D work adds specialized coordinate, view, and linkage constraints; interactive applications add browser behavior and state.
- Optimize for
- fidelity · edit locality · reproducibility · behavior
- Human control
- varies with domain risk and reversibility
What is being compared
Next, separate papers, tools, skills, benchmarks, and products.
They overlap, but they are not interchangeable. A paper proposes and evaluates a method. A tool is runnable software. A skill guides a general agent. A benchmark supplies tasks and scoring. A product operates inside a vendor’s data environment.
Object
Claim it makes
What it can establish
Examples
Research papermethod + study
“Here is an approach, and here is how we tested it.”
Usually a prototype plus an experiment. The result applies to the paper’s tasks, models, baselines, graders, and study population—not automatically to production.
Examples: Raiven, nvAgent, DashChatOpen-source toolrunnable code
“Here is software you can inspect or operate.”
The repository shows current mechanisms, interfaces, tests, and failure handling. Code quality and adoption do not by themselves prove better analysis.
Examples: Data Formulator, Lumen, LIDAAgent skillworkflow package
“Here is how a general agent should perform this job.”
Instructions, references, templates, and sometimes check scripts. The skill is not a new model; it changes the model’s working context and required process.
Examples: Vizro flow, OpenAI visualize-data, AntV skillsBenchmarktasks + grader
“Here is a repeatable way to compare systems.”
A benchmark fixes the inputs and scoring rules. It can expose relative strengths while still missing production data, real readers, or consequential decisions.
Examples: DashArena, VisEval, MatPlotBenchCommercial productnative assistant
“Here is an AI feature inside an existing analytics system.”
Its advantage often comes from access to governed fields, measures, permissions, and authoring state. Feature documentation is not an accuracy study.
Examples: Power BI, Tableau, LookerHow the pieces fit
Creation and evaluation are separate jobs.
A skill may guide an authoring agent. The agent may emit code directly or first write a structured plan. A renderer turns that work into the chart or dashboard. Checks, benchmarks, and human studies then answer different questions about the result.
The core system papers, plainly
They study different parts of the pipeline.
These eight papers anchor the architecture comparison. Only DashArena is primarily a benchmark. Additional benchmark and training studies are introduced later where they answer the separate question of how quickly the model baseline is changing.
Data Formulator 2
What it is: a chart interface where a person directly picks visual encodings and asks the model for missing data transformations.
Why it matters here: it shows how visible tables, direct controls, branches, and explanations help people steer and verify model work.
Raiven
What it is: a prototype where the model writes a restricted visualization language and a compiler produces linked 2D, 3D, and table views.
Why it matters here: it is strong bounded evidence that a DSL and compiler can remove many coding failures when the requested view is already specified.
NL2Dashboard
What it is: an architecture that stores a dashboard as a compact structured plan and applies small edit operations instead of regenerating everything.
Why it matters here: it tests whether explicit structure improves controllability, edit locality, and token use.
nvAgent
What it is: a pipeline that prepares a database schema, composes a visualization query, executes it, and repairs failures.
Why it matters here: its large VisEval benchmark and component removals help isolate structured composition and execution validation.
DashChat
What it is: a conversational mockup tool with dashboard-pattern retrieval, a restricted dashboard language, structured edits, and history.
Why it matters here: it combines a prompt evaluation with a small study of rapid prototyping and human correction.
NL4DV-LLM
What it is: a method that turns a natural-language question into selected fields, an analytical task, and one or more candidate visualizations.
Why it matters here: it makes mappings and ambiguity visible instead of jumping straight from a sentence to chart code.
PlotGen
What it is: a Matplotlib pipeline with separate numeric, text, and rendered-image feedback passes around generated code.
Why it matters here: it supports multimodal feedback while showing why extra agents and calls need an equal-budget causal test.
DashArena
What it is: a task set, browser executor, and human-calibrated judge for generated interactive dashboards—not an authoring product.
Why it matters here: it evaluates the rendered dashboard and replays the author’s intended interactions, filling a major gap between “it runs” and “it supports the task.”
Terms used later
A short glossary
- Agent
- A model operating in a loop with instructions, tools, and working state—not a synonym for any AI feature.
- IR
- Intermediate representation: a structured, inspectable plan between a request and renderer code.
- DSL
- Domain-specific language: a restricted language for one problem area that a compiler can interpret reliably.
- Semantic model
- A governed description of fields, measures, joins, permissions, and business meaning above raw tables.
- Renderer
- Software that turns a specification or code into pixels and interactive controls.
- Critic
- A model pass that inspects code or a rendered artifact and proposes faults or repairs.
- Trajectory
- A recorded sequence of interactions and resulting states that a browser can replay.
- Held-out
- An evaluation task not used as an example while building or prompting the system.
- Ablation
- A controlled comparison that removes or changes one component to test whether it caused the gain.
- Mixed initiative
- A workflow where natural language and direct human controls both change the artifact.
Research synthesis
What current evidence establishes—and what it does not.
This synthesis distinguishes repeated findings from environment-specific choices, direct negative or null evidence from merely unsupported claims, and demonstrated capability from questions the literature still does not answer.
Synthesis
The literature agrees on five practices, not one system design.
Across the papers, tools, skills, and products, the most consistent pattern is to reduce what the model must improvise, expose consequential choices, and test the artifact at the layer where a failure can occur. The particular representation and interface still depend on the environment.
Technique area
Where evidence converges
Where design depends on context
What performs poorly or proves too little
What remains open
Intent and representation
Expose fields, transforms, encodings, interactions, and edits in an inspectable plan, language, contract, or native semantic model.
A compiler-backed DSL fits stable scientific grammars; direct generation may be proportionate for reversible one-off charts; governed BI can rely on maintained measures.
A formally valid plan can encode the wrong question. In Raiven, the DSL advantage was concentrated in scientific visualization rather than ordinary information graphics.
Which representation earns its cost for each environment, especially as direct model generation improves?
Verification and evaluation
Use computation for values, joins, filters, and runtime claims; evaluate the rendered artifact and, when interactive, its behavior and resulting state.
Static work may stop at final-context render inspection. Interactive work needs action coverage and replay. High-stakes work needs independent acceptance.
A nonblank render or clean execution is weak evidence: 21 of 30 cleanly executed DashArena failures still had semantic defects.
How should benchmark evidence connect to comprehension, retention, calibration, and real decisions?
Human steering
Visible transformed data, direct controls, history, branching, and reversion help people inspect and correct model work.
Exploration needs continuous steering; editorial explanation needs authorial control over claim and sequence; monitoring needs governed thresholds and response paths.
Synthetic personas are not reader evidence. A model-authored interaction trajectory describes intended use, not observed human behavior.
Do generated visuals help a defined audience understand, remember, or decide better over time?
Roles, feedback, and instructions
A separate role is defensible when it has different information, tools, or independent evidence: schema access, execution, visual inspection, or browser replay.
One capable model may be better for simple work; specialized roles may help when failure layers and evidence sources are genuinely distinct.
Repeated critics show diminishing returns. More instruction can reduce execution. Removing nvAgent’s processor slightly helped GPT-4o overall while hurting weaker and multi-table cases.
Equal-budget tests of role decomposition and behavioral with-and-without tests of skill packages remain rare.
Grounding and evidence claims
Source custody, semantic definitions, permissions, denominators, and data vintage must travel with the artifact.
The grounding source differs: governed semantic models in enterprise BI, explicit source packets in editorial work, and domain types in scientific systems.
Repository popularity, install count, saved API responses without renders, one vision-model score, or a validator that fails open cannot establish quality.
Public product evaluations stratified by semantic-model quality and realistic data conditions are still missing.
Shared timeline · September 2023–August 2026
From 2023 to 2026, the test expanded from isolated charts to languages, documents, and dashboard state.
Model and harness releases expanded what a general system could see and do. Research systems added structured plans, direct editing, compilers, and feedback. Benchmarks added real data, multi-turn refinement, professional domains, deception, chart pairs, multi-chart figures, multilingual reading, document retrieval, browser behavior, and analytical use.
Model and harness
Research system
Benchmark
Model and harness baseline
What a general agent can see, execute, and reuse
- GPT-4V adds image input
General models can inspect visual artifacts.
- coding agent · agent tools · GPT-5
Terminal coding, tools, tracing, and stronger agentic coding enter normal harnesses.
- Hosted execution and reusable skills
Shell, persistent workspaces, reusable skills, and compaction become platform primitives.
Research systems and techniques
How systems structure authoring and correction
- Data Formulator 2 · NL4DV-LLM
Visible transformed data, direct controls, history, and explicit analytic specifications.
- PlotGen · DashChat · nvAgent
Multimodal feedback, dashboard languages, schema planning, execution, and repair.
- NL2Dashboard · Raiven
Compact dashboard plans with atomic edits; restricted scientific language with deterministic compilation.
Benchmarks and measured capability
What the field can test repeatably
- Plot2Code · VisEval
Plot reproduction and database visualization, with execution and visual checks.
- Text2Vis
1,985 tasks join data, questions, answers, code, and annotated charts; targeted review improves pass rate.
- RealChart2Code · Chartography · ChartDiff · PolyChartQA · Chart-MRAG · DashArena
Real data, professional reading, chart pairs, multi-chart and multilingual QA, document retrieval, state, and browser replay.
What the benchmarks actually measure
A benchmark result is a task-and-grader result, not a field-wide score.
The same system can pass code checks, misread the finished chart, miss a deceptive axis, or fail after a dashboard control changes state. Read the evidence as cumulative obligations. No captured evaluation spans the whole episode.
- 01
Structure
Requested code, fields, marks, layout, or specification
- 02
Execution
Compile, render, load, and callback completion
- 03
Semantics
Values, transformations, bindings, and encodings
- 04
Perception
Extraction, grounding, questions, and comparison
- 05
Integrity
Deception, defects, and calibrated quality judgment
- 06
Interaction
Control behavior, hidden state, and intended-use replay
- 07
Human outcome
Accepted work, comprehension, decisions, and access
Evaluation family
What the system must do
Representative evidence
What it still cannot establish
Creation
Generation and reconstruction
Turn an image, source data, database question, or specified view into chart code, a visualization query, or a compiled artifact.
Plot2Code · VisEval · Text2Vis · RealChart2Code · Raiven
Execution and resemblance do not establish that the question, source, semantics, or design is appropriate for a person.
Reading
Perception, reasoning, and integrity
Extract values, locate marks, answer questions, detect deceptive encodings, or predict an expert-defined quality rating.
ChartQAPro · Misviz · VisJudge · FinChart-Bench · Chartography
Chart reading is not chart authoring. A visual critic usually cannot verify raw data, audience fit, interaction, or reader outcome.
Context
Multiple charts, documents, and languages
Compare chart pairs, localize and reason across multi-panel figures, retrieve charts and text from documents, or work outside English.
ChartDiff · multi-chart PolyChartQA · Chart-MRAG · multilingual POLYCHARTQA
Translated, generated, or carefully curated sources do not represent ordinary work across languages, institutions, devices, or local chart conventions.
Use
Interaction and people
Navigate, reconstruct, or generate an interactive dashboard; replay intended use; or observe a creator or reader performing a task.
DashboardQA · Dashboard2Code · DashArena · bounded authoring and reader studies
No evaluation follows one artifact through authoritative data, acceptance, delivery, mobile and assistive use, reader outcome, later correction, and maintenance.
Up to 27.4 points lower
Multi-chart accuracy fell on human-authored questions compared with model-generated questions.
ROUGE and human-aligned quality disagreed
ChartDiff specialists and pipelines scored higher on overlap while general models led on calibrated quality.
+15.4–19.6 on target cases; as low as −8.5 elsewhere
Table-based QA resisted misleading charts but sometimes damaged accuracy on non-misleading charts.
A moving model-and-harness baseline
Models are improving, so generic instructions expire faster than local context and checks.
There is no clean visualization-specific curve covering August 2023 through August 2026. Benchmarks, judges, models, prompts, and test-time budgets changed. The literature supports a direction and several measured slices—not a universal annual improvement rate or a defensible straight-line forecast.
How to read the evidence
Improvement is large in fidelity and breadth, smaller in basic execution, and uneven by task.
The best comparable results come from papers that evaluate multiple model generations on the same benchmark. Even those slices are not a pure measure of training progress: provider, model size, vision stack, inference policy, and evaluator can all change together.
The widening task envelope is just as important as the score movement. In 2023, a demanding benchmark asked a model to reproduce one scientific plot. By 2026, DashArena asks a general model to construct a multi-view interactive dashboard and declare a replayable analysis path. Broader capability does not mean dependable behavior.
How to read the paired values. Each row has its own labeled scale. Compare the open earlier point with the filled later point inside that row; horizontal positions are not comparable across different metrics.
Twenty-one months of movement on one scientific chart-to-code benchmark
An anonymous 2026 preprint re-reported the same Python/Matplotlib subset and normalized the old and new model results over the full test set. Useful trend evidence; not peer-reviewed longitudinal proof.
A later general model improves quality more than execution
ACL 2026. The chart-specialized CharTide models also matched or exceeded general frontier models, showing that better baselines can come from domain training as well as scale.
Targeted feedback helped
Text2Vis: 26% direct → 42% with one answer-and-code review
Three examples left GPT-4o at 26%; retrieval plus examples reached 31%. One structured answer-and-code feedback round reached 42%. Adding visual feedback improved visual subscores but slightly reduced final pass to 41%.
Interpretation: add the missing evidence channel; do not equate another prompt or model call with progress.More instruction sometimes hurt
Plot2Code: stricter requirements traded execution for resemblance
Detailed conditional instructions generally improved similarity while lowering code pass rates; Gemini Pro fell from 68.2% to 55.3%. Chain-of-thought and Plan-and-Solve showed no clear advantage over the default prompt.
Interpretation: prose scaffolding can collide with the task, budget, or model rather than simply add knowledge.Breadth outran reliability
DashArena: full dashboard generation is now plausible, not dependable
Current models can generate an interactive dashboard and intended-use trajectory from an open task. The strongest aggregate preference result was competitive with the human baseline, but no model cleared 86% rendering or 74% replay.
Interpretation: the frontier moved from “can it attempt this?” to “where and how does it silently fail?”State is part of correctness
Dashboard2Code: complex interaction remains much harder
The best configuration scored 79.4 overall and 64.2 on the hardest interaction level across 180 Plotly Dash applications. DOM access raised one model’s successful UI exploration from 20.61% to 64.58%.
Interpretation: a dashboard can respond while hidden state or a transformation is wrong. Fixed 1920×1080 desktop, no animation or popups.Professional reading is a separate test
Chartography: the best configuration passed 45% of deliberately hard tasks
One hundred practitioner-authored tasks span twelve domains and have three expert verifiers each. More reasoning usually helped, but often elaborated an initial visual misread.
Interpretation: this adversarial slice is not a failure rate for ordinary charts; it shows why basic chart-QA success is not a professional release gate.Preference did not imply accuracy
BLV learners preferred tactile + text + chat, with no measured accuracy lift
Eleven of twelve participants preferred the multimodal condition and described a better spatial mental model. The chart-understanding accuracy comparison did not improve.
Interpretation: measure modality, mental model, preference, and comprehension separately.Forecast from the August 2026 evidence cut
The next unlock is a chain of evidence, not a better first draft.
A chart capability becomes useful as it moves from grounded intent to inspectable construction, verification, repair, delivery, and reader outcome. A stronger model can move several links. It cannot replace evidence the system never sees.
- 01
Ground
Question, definitions, source, audience, stakes
- 02
Construct
Visible transforms, semantic state, alternatives
- 03
Verify
Source, values, render, interaction, delivery
- 04
Repair
Fix locally without introducing a regression
- 05
Deliver
Browser, mobile, accessibility, handoff, update
- 06
Help
Readers understand, decide, learn, or act better
Why the horizons differ
Observable, executable failures are likely to improve first.
Static chart generation, dashboard interaction, visual tools, and narrow adapters already have public tests and measurable gaps. Production productivity and reader benefit require field evidence that is mostly absent.
The percentages are dated probabilities that a declared public evidence test will pass—not estimates of how intelligent a future model will be. Confidence is lower where the field has no stable base rate.
Evidence thresholdByProbability
Reliable static work
At least 70% accepted success on 500+ real-data chart tasks with executable and human-calibrated visual checks.
78%56% confidence
Interactive dashboard reasoning
More than 60% on DashboardQA or a harder successor through executed, replayable interactions.
64%52% confidence
Critic or tool layer that repairs
An independent same-model test adds ten points to detection or repair without lowering total acceptance.
72%59% confidence
Production productivity
Two environments show 20% less total human time to an accepted, maintainable artifact without quality loss.
43%42% confidence
Reader benefit in consequential use
Two contexts improve representative-reader comprehension or calibrated trust over human-only professional production.
34%37% confidence
Autonomous publication
Three environments clear source, interaction, mobile, accessibility, reader, and update gates without human acceptance.
14%34% confidence
The evidence ladder
“Works” is seven different claims.
Passing a lower layer is necessary. It is never a proxy for all the layers above it.
-
01
AntV · OpenAI skill · Vizro not execution
Structure
Required fields, files, or spec entries exist.
-
02
Raiven · Vizro · DashArena not correct values
Execution
Code runs and a nonblank artifact appears.
-
03
nvAgent · Raiven · Vizro not legibility
Data fidelity
Values, joins, filters, and aggregations match the source.
-
04
Raiven · PlotGen not insight
Visual integrity
Encodings are coherent and not misleading.
-
05
DashArena · Vizro not a good analysis
Interaction
Controls and linked views behave as intended.
-
06
DashArena not learning or decisions
Analytical support
The artifact helps address the task.
-
07
Open in this scan requires real readers
Reader outcome
A defined audience understands, remembers, or decides better.
What changed this month
DashArena tests whether a dashboard supports analysis—not whether a reader learns.
Its judge sees the task, schema, screenshots, and a browser-replayed interaction trajectory. Adding interaction evidence raised agreement with dashboard-experienced humans by 8.1 percentage points.
Earlier wording
→
No benchmark measures communicative value.
Better wording now
Task-grounded analytical support has a credible measure. Reader comprehension, retention, and decision quality remain open.
Full evidence
79.8%
No interaction evidence
71.7%
Rules only
DashArena · 100 labeled pairs; 99 evaluable · six dashboard-experienced annotators
42.4%
Why a clean render is weak evidence
Failure survives at every layer.
No tested model exceeded 86% render success or 74% trajectory replay success. Among 30 execution-clean failures, 21 still contained semantic defects.
- Construction / runtime26.9%
- Interaction25.4%
- Data computation / binding22.4%
- Presentation / readability16.4%
- Analytical9.0%
Rounded share of a targeted 100-failure audit. The categories partition the audited failures; they are not prevalence estimates for all generated dashboards.
Technique map
Five techniques recur in the strongest results.
The recurring pattern is a controlled path from ambiguity to evidence. Different environments need different pieces of it.
Constrain
Typed intent or visual IR
Put fields, transforms, encodings, layout, and interactions between natural language and renderer code.
A/B bounded Raiven · nvAgent · NL2Dashboard · NL4DV-LLM · Flint
Can formalize the wrong question or exclude useful forms.
Externalize
Visible state and reversible edits
Keep transformed tables, direct encoding controls, branch history, explanations, and atomic modifications in view.
A small studies Data Formulator 2 · DashChat
Learnability and prototype speed are not long-term correctness.
Ground
Semantic and data contracts
Bind questions to governed measures, field metadata, permissions, verified queries, sample values, and source vintages.
C/D mechanism Power BI · Tableau · Looker
A semantic layer can be stale, wrong, or silent about missing concepts.
Verify
Deterministic work for deterministic claims
Compile, recompute, inspect ASTs, test aggregation and encoding invariants, render in a browser, and record runtime evidence.
B/C repeated Raiven · nvAgent · Vizro · DashArena
Check coverage is partial; “no exception” is not semantic truth.
Observe use
Replay and human acceptance
Give evaluation the rendered artifact, intended task, interaction sequence, resulting state, and a defined human judgment.
A/B frontier DashArena · Data Formulator · DashChat
Model-authored intent is not actual reader behavior.
Choose the environment first
Different environments need different controls and tests.
A compiler can prevent coding failures in a stable scientific grammar; it may be needless overhead for a one-off chart. Compare environments here before interpreting the systems that follow.
EnvironmentInvest inHuman keepsExamples
Scientific / 3D
Invest in domain DSL, compiler, coordinate and link types, reproducible transforms
Human keeps scientific interpretation and misleading-but-legal review
RaivenGoverned enterprise BI
Invest in semantic models, verified queries, permissions, source metadata, native editing
Human keeps metric ownership, source validation, and consequential interpretation
Looker · Power BI · TableauExploratory analysis
Invest in visible transformed data, direct encodings, short NL transforms, branches, reversion
Human keeps question choice, correction, and stopping judgment
Data FormulatorEditorial explanation
Invest in chart contract, evidence custody, denominators, final-context inspection, accessibility
Human keeps claim, audience, integrity, acceptance, and publication
High-context caseCoding-agent dashboard
Invest in versioned retrieval, specs, running app, action coverage, console/network receipts
Human keeps acceptance against the task and uncovered interaction risk
Vizro + DashArenaOne-off chat / Markdown
Invest in thin renderer adapter and reversible output when data and intent are simple
Human keeps awareness of what was rendered and what was never validated
Vega skillResearch and implementation landscape
The same techniques appear in research systems, skill packages, and BI products.
Read each row across: mechanism, strongest evidence, and limit stay aligned. Evidence grades are not product scores: A marks comparative human evidence, B a comparative benchmark, C an inspectable implementation, and D a product or author claim.
Show
System or package
Mechanism
Strongest evidence in this review
Limit or missing evidence
Source
Open-source tool + CHI paper
Data Formulator
Natural language for transformation; GUI for explicit encodings; visible tables, code, branches, and reversion.
Best signal Eight people reproduced 16 charts and 12 nontrivial transforms.
Supports authoring and verification behavior—not long-term analytical correctness.
Repository ↗Research prototype + preprint
Raiven
Metadata → RaivenDSL → deterministic compiler for linked 2D, 3D, and tabular views.
Best signal 100% compile and .988 VMPC on 100 fully specified prompts.
Most advantage is in SciVis; the task specifies the target view rather than discovering the question.
Paper ↗Research architecture + preprint
NL2Dashboard
Compact IR separates analysis, content, and layout; atomic edits avoid full-dashboard rewrites.
Best signal Completed all edit tasks across seven modification classes with low output-token ratios.
Ten tables, LLM judge, unmatched model interfaces, and no user study.
Paper ↗Research system + ACL paper
nvAgent
Schema processor, sketch-and-fill VQL composer, and execution-guided validator.
Best signal +7.88 / +9.23 pass-rate points on single- / multi-table VisEval.
The composer drives most of the gain; processor value depends on model and environment.
Paper ↗Research prototype + preprint
DashChat
Pattern retrieval, a DSL, focused calls, chat plus structured edit bubbles, and visual history.
Best signal 100% executable and 94% spec consistency on 50 prompts; 28-person study.
Mock-data prototype tasks do not establish production analytics quality.
Paper ↗Open-source framework
Lumen
Coordinator routes to SQL, Vega-Lite, Deck.gl, source, table, and validation agents over serializable pipelines.
Best signal Inspectable implementation, deterministic profiling, and real capability boundaries.
No persuasive comparative efficacy study; one inspected validation path can fail open.
Repository ↗Open-source research toolkit
LIDA
Summarization, persona-conditioned goals, chart-code generation, model scoring, repair, and recommendation.
Best signal Influential inspectable baseline for a modular prompt pipeline.
The evaluator does not inspect the final rendered image or create regression evidence.
Repository ↗Research method + preprint
PlotGen
Planner and code generator followed by numeric, lexical, and visual feedback agents.
Best signal 65.67 on MatPlotBench versus 61.16 and 48.86 baselines.
No equal-budget baseline; heavy VLM use and reporting inconsistencies obscure the causal mechanism.
Paper ↗Research method + preprint
NL4DV-LLM
Model emits an inspectable analytic specification and can preserve multiple interpretations of ambiguous prompts.
Best signal 87.02% across 740 queries versus 64.05% for rule-based NL4DV.
Older-model evidence; valid syntax does not guarantee correct binding.
Paper ↗Benchmark + browser executor + judge
DashArena
Agent authors a dashboard and interaction trajectory; Playwright replays it; a calibrated judge sees task, schema, states, and execution evidence.
Best signal 234 tasks / 14 clusters; interaction evidence adds 8.1 points of human agreement.
Tableau-seed-biased, partial authored trajectories, and no reader-outcome measure.
Paper ↗Skill package inside an open-source tool
Vizro flow
Six staged skills, required specs and tests, AST checks, action enumeration, Playwright, and a test receipt.
Best signal Strongest mechanics of the skill packages inspected.
Seven fixtures and no aggregate held-out or independent human outcome.
Package ↗Agent skill package
OpenAI visualize-data
Question and takeaway first, explicit chart contract, data sufficiency, delivery routing, and final-context QA.
Best signal Broadest inspected contract and delivery checklist.
No behavioral comparison of an agent with and without the skill.
Skill ↗Agent skills + reference corpus
AntV skills
Strict G2/G6/X6 version constraints plus hybrid retrieval over a large reference corpus.
Best signal Largest included retrieval and code benchmark in the inspected skills.
Saved runs did not include render results; “success” mostly means a response completed.
Package ↗Skill family + paired evaluation
SciVisAgentSkills
Version-pinned guides for four scientific tool families, with headless execution and render–inspect–adjust loops.
Best signal Quality improved in all ten suite-by-agent comparisons across 108 tasks.
Author-built and author-evaluated, scientific tasks only; one completion measure fell and no independent or reader-outcome test was found.
Study ↗Two instruction-only skill packages
Vega + chart primers
Renderer syntax and examples in Markdown Viewer; long chart-literacy and accessibility prose in claude-skillz.
Best signal Useful surface adapters and a clean prompt-only baseline.
No fixtures, data checks, render loop, or outcome evaluation.
Vega skill ↗Commercial BI feature
Power BI Copilot
Selects tables, fields, measures, and charts from a semantic model; output remains editable in native tools.
Transfer Bind generation to governed measures and retain direct author control.
Current capability documentation, not comparative accuracy or user-outcome evidence.
Documentation ↗Commercial BI feature
Tableau Agent
Uses connected-source fields, metadata, sample values, and worksheet state; results stay directly editable.
Transfer Keep agent scope close to native authoring state and make direct manipulation the recovery path.
The vendor says to review results and documents material current limitations.
Documentation ↗Commercial BI feature
Looker Conversational Analytics
Grounds requests in LookML, permissions, descriptions, samples, glossaries, and optional verified queries.
Transfer Maintained semantic custody beats a generic analyst persona.
No public evidence here isolates which grounding feature improves decisions; outputs still require validation.
Documentation ↗Skill-package maturity
One specialist family reaches paired proof; most packages do not.
- Adviceprose and examples
- Retrievalversioned references
- Artifactscontract and specs
- Checksdata and source
- Browserrender and actions
- Receiptsevidence retained
- Ablationwith / without skill
- Outcomehuman benefit
SciVisAgentSkills reaches paired ablation for specialized scientific work, but not independent reproduction or a human outcome. Vizro reaches furthest on execution evidence; AntV has the largest retrieval/code benchmark; OpenAI has the broadest contract and delivery QA. Those are different strengths—not one winner.
Research gap ledger
A gap can be narrowed without being solved.
Each row states what has actually been demonstrated, what remains missing, and the evidence that would move the field. “Partly answered” means one bounded study exists—not that the result transfers to ordinary production.
GapEvidence nowStill neededStatus
One-pass competence
Realistic chart and dashboard benchmarks show materially stronger generation alongside render, replay, and semantic failures.
A stratified sample of editorial, operational, scientific, and governed work scored through human acceptance.
Measured, boundedSemantic fidelity
Several systems test data binding or calculation in bounded grammars; execution-clean dashboard outputs can still contain semantic defects.
Accuracy against owned measures, ambiguous fields, changing sources, permissions, and unstated local rules.
Measured, boundedCorrection without regression
Multi-turn research records regressive editing; in one novice study most observed clutter and unusable-chart repairs failed.
Accepted-artifact correction cost, abandonment, reliable stopping, and regression after delivery.
Failure demonstratedCritique and specialized vision
A trained 7B critic beats tested general models on one expert-adjudicated quality rubric; chart-specific tools improve numeric QA and grounding. Strong general models lead some newer transfer tests.
End-to-end, equal-budget routing tests over source fidelity, integrity, readability, interaction, mobile states, repair success, and reader outcomes.
Partly answeredAuthoring and expertise
Controlled novice work, a 13-designer several-day critique study, and a three-person enterprise probe expose different benefits and failure detection.
Representative current-system work followed through publication, organizational review, and later correction.
Partly answeredReader outcomes
A randomized 117-person educational study favors proactive scaffolding for post-support comprehension; synthetic personas fail as reader substitutes.
Generated-artifact comprehension, retention, decisions, calibrated trust, accessibility, mobile reading, and harm.
Partly answeredInternational and non-English use
Multilingual chart QA covers ten languages, while a chart-centric judge benchmark spans 25. Both find language-dependent accuracy or bias.
Human-authored local work, cultural chart conventions, code-switching, more low-resource languages, and delivered reader outcomes. Current sets translate English-centric sources.
Partly answeredSkill-package lift
One scientific skill family improved quality in all ten paired suite-by-agent comparisons; broader skill benchmarks show conditional gains and regressions.
Independent tests of popular generic, dashboard, accessibility, mobile, and explanatory skills against the current model and normal harness.
Partly answeredContext-sensitive routing
Task and environment typologies explain why journalism, BI, exploration, science, operations, and education differ.
Which dimensions actually change the best generator, critic, evidence bundle, or human gate.
OpenDelivered interaction
Browser replay and visible interaction evidence improve dashboard evaluation.
Real-user exploration, authenticated apps, permissions, exports, assistive technology, devices, and recovery paths.
Measured, boundedMaintenance and total cost
Papers report fragments such as inference budget, iteration count, or study time.
Human work, waiting, verification, failed repair, model cost, deployment, abandonment, and future updates on one ledger.
OpenReproducibility and transfer
Some studies release code, tasks, or supplements; others retain executors, judges, model versions, or failure traces.
Independent reruns, cross-renderer tests, stable tasks, and calibration against readers or domain experts.
OpenMethod and custody
What we reviewed—and what we did not test ourselves.
The report uses pinned repositories, papers, and first-party product documentation. It reports their evidence without claiming that their experiments were independently reproduced.
Eight open-source repositories and eight core system papers in the initial review; additional generation, professional-reading, integrity, multi-chart, multilingual, document-retrieval, interactive-dashboard, and human-outcome studies; three product surfaces, seven model-or-harness milestones, and three established visualization frameworks. Companion deep dives add a task-and-grader benchmark crosswalk, 18 skill files or families, broader skills benchmarks, and the paired SciVisAgentSkills study.
No external repository, package, installer, skill, model gateway, or untrusted script was run. Included tests and results are observations from the sources, not independent reproductions.
Mechanism, evidence layer, environment fit, and failure boundary are valid comparisons. Cross-benchmark score ranking and popularity are not.
Recheck by 14 Nov 2026—or earlier after a material model, harness, benchmark, registry, or analytics-assistant release.
Primary sources inspected Pinned revisions and publication records
Pinned repositories
Papers read in full
- DashArena · 2026
- Raiven · 2026
- NL2Dashboard · 2026
- nvAgent · ACL 2025
- DashChat · 2025
- Data Formulator 2 · CHI 2025
- NL4DV-LLM · 2024
- PlotGen · 2025
Moving-baseline studies
- Plot2Code · Findings of NAACL 2025
- Text2Vis · EMNLP 2025
- CharTide · ACL 2026
- RRVF · anonymous ICLR 2026 submission
- Dashboard2Code · 180 dashboards · 2026
- Chartography · 100 professional tasks · 2026
- POLYCHARTQA · ten-language chart QA · ACL 2026
- Chart-MRAG · chart-bearing documents · ACL 2026
- ChartDiff · chart-pair comparison · ALVR 2026
- Beyond Single Plots · multi-chart QA · Findings 2026
- FinChart-Bench · financial chart comprehension · ACL 2026
- Misleading-visualization corrections · ACL 2026
- MM-JudgeBench · multilingual visual judges · Findings 2026
Skill-package evidence
Human-outcome and situated studies
- Vibe Visualizing · 20 novices · 2026
- Visualizationary · 13 designers + three raters · 2024
- Visual-comprehension scaffolding · 117 participants · 2024
- AI-supported end-user development · eight interviews + three probes · 2025
- Touching or Chatting · 12 BLV participants · 2026