The state of AI-assisted data visualization research

Status: research snapshot, evidence cut 2026-08-18. Recheck by 2026-11-16, or earlier after a material model, harness, benchmark, registry, or analytics- assistant release.

Short version: executive summary. Forward view: What would unlock the next capabilities in AI-assisted data visualization? turns the moving-baseline evidence into dated, scoreable forecasts and keeps model-driven and technique-driven gains separate. Assessment view: How close is AI-assisted data visualization to the ideal? defines a context-sensitive ideal, scores nine current capabilities on one fixed maturity scale, reconstructs the same scorecard from 2017 through August 2026, and translates the forecasts into explicit score movements.

This report is for researchers, product teams, designers, and engineers who need to understand what AI-assisted visualization systems can currently do, which techniques appear to improve them, and how those claims are measured. It is a research synthesis and mechanism comparison, not a product ranking or buyer’s guide. Tools, agent skills, and commercial BI assistants appear as examples of techniques in practice; their documentation does not establish comparative quality.

Eight open-source repositories were inspected statically at the pinned revisions listed below. Eight core system papers were read in full, four initial benchmark and training studies were read for the moving-baseline analysis, three current commercial product surfaces were checked against first-party documentation, seven first-party model and harness release records were used for the shared timeline, and earlier papers in the underlying research packet were re-read where they controlled a comparison. Three established visualization frameworks were added to ground the distinction between task goals, narrative explanation, dashboards, and exploratory tools. A companion deep reading adds 18 agent-skill files or families, broader skills benchmarks, and the paired SciVisAgentSkills study; its report keeps registry installations, repository popularity, and behavioral evidence separate. A second specialized-vision report separates chart QA, parsing, OCR/layout, grounding, quality or integrity critique, and verifier/repair roles rather than treating every visual model as an interchangeable judge. A third companion, Learning data visualization when AI can make the chart, separates assisted performance from retained skill, distinguishes productive difficulty from removable friction, and forecasts how human practice should be reallocated as implementation becomes cheaper. A fourth companion, What AI-assisted data visualization benchmarks actually measure, translates benchmark names into their inputs, outputs, source distributions, graders, human context, supported claims, and nonclaims. No external repository, package, installer, skill, model gateway, or untrusted script was run. Repository tests and included evaluation results are observations about what the source contains, not independently reproduced results.

Start here: what agentic data visualization is trying to do

Agentic data visualization is not one task. A system may help a person discover a pattern, explain a finding, monitor a changing situation, make a decision, or produce and revise a durable visual artifact. These purposes can occur in a sequence—exploration may eventually become a published explanation—but they do not have the same success condition.

Rather than invent a new top-level taxonomy, this report borrows the discipline of Brehmer and Munzner’s established visualization-task typology: describe why the work is undertaken, what data and outputs it acts on, and how encoding and interaction support it. This matters acutely for agents. Returning a technically valid chart answers part of “how”; it does not show that the system served the intended human purpose.

These are overlapping goals, not product bins. Segel and Heer characterize narrative visualization by the balance between author-driven narrative flow and reader-driven discovery. Sarikaya and colleagues show that dashboards also vary by purpose, audience, interaction, and data semantics, and differ materially from exploratory visualization tools. A newsroom interactive may contain both guided explanation and exploration; a dashboard may monitor, analyze, teach, or communicate. The important step is to declare the sequence and apply the right evidence at each transition.

That produces a general requirement: every system should preserve the question, data, decisions, artifact, and evidence. A context contract should then select the appropriate authoring and evaluation method. A universal generator, critic, or acceptance score would erase differences the system needs to understand.

What is being compared

This landscape contains several kinds of thing that are easy to blur together. A paper can describe a tool, a repository can implement a paper, and a benchmark can be used to test several tools. They still make different kinds of claim.

The objects fit together roughly like this, but the short pipeline hides who may decide what:

question + data
      |
      v
authoring system or product  <--- an agent skill can guide this work
      |
      v
structured plan, DSL, or code
      |
      v
renderer + browser  ---> executed chart or interactive dashboard
      |
      +--- deterministic checks test values and behavior
      +--- a benchmark supplies repeatable tasks and scoring
      +--- a human study tests how people author, understand, or decide

A reference anatomy assigns information and authority

A bounded audit of six held primary authoring or analysis systems—Data Formulator 2, Raiven, MatPlotAgent, Text2Vis, ViviDoc, and interactive task decomposition—supports an eleven-stage reference anatomy. The stages are a custody checklist, not one required agent topology:

The six-system audit finds 6/6 generation, execution, and some critique route; 4/6 meaningful human-control gates; and 0/6 exact-configuration accepted delivery, intended-reader outcome, or later maintenance. The 6/6 critique count is not architectural convergence: typed validation, runtime errors, visual-model feedback, and human inspection see different evidence and hold different authority. VisJudge can add one specialist critique signal; DashboardQA can test downstream agent interaction. Neither closes the human gates.

Primary anchors: Data Formulator 2, Raiven, MatPlotAgent, Text2Vis, ViviDoc, and interactive task decomposition.

Public acquisition is not adoption

The same custody rule applies to agent-skill popularity. Twelve named data-visualization registry listings carry 37,720 listing-summed install signals at the 14 August evidence cut. They collapse to ten parent repositories and eleven documented provenance lineages. The sum is not a count of unique people, teams, successful installations, invocations, or outcomes.

The registry calls its field a total deduplicated install count but does not publish the deduplication key or interval. At a pinned official CLI commit, an install event is assembled from selected skills after target results are collected; telemetry may be disabled or absent, and its schema contains no skill-invocation, task-outcome, retention, or organizational-acceptance event. The private audit therefore finds 12/12 public install signals and 0/12 held rows with observed real-task invocation, repeat use, organizational acceptance, or outcome/afterlife. Those zeros are bounded to the named public surfaces; private use remains unknown.

Use the skill-package report for the package-level table and evidence boundary. Use registry counts to choose what to inspect, not to make an adoption, procurement, or effectiveness claim.

Available controls are not governed deployment

The organizational-governance evidence also separates into two rails. Power BI/Fabric, Tableau, and Looker document meaningful enablement, data-boundary, and monitoring controls. Provider stories separately report named-feature use of Looker Conversational Analytics at Google Cloud Support and Copilot in Power BI among KPMG developers. Two longitudinal studies add adjacent governance- process evidence. Across those seven held rows, zero joins the complete same-deployment record.

The join requires one feature and version, organizational authorization, enablement and scope, data or semantic authority, monitoring and audit, routine use, incident or exception disposition, an independently accepted outcome, and a later recheck. Provider controls from one row cannot be spliced to outcomes from another. This is a bounded public-surface result, not evidence that no organization holds a stronger private record. The practitioner account contains the provider-control, organizational-use, and source boundaries.

The eight core papers are not eight versions of the same product. Additional benchmark and training studies are introduced later where they answer the separate question of how the model baseline is changing.

Terms used throughout the report:

• Intermediate representation (IR): a structured, inspectable plan between a request and renderer code—for example fields, transforms, encodings, layout, and interactions stored as JSON-like data.
• Domain-specific language (DSL): a restricted language for one problem domain. It gives up some freedom so a compiler can reject or interpret output more reliably.
• Semantic model or semantic layer: a governed description of fields, measures, joins, permissions, and business meaning that sits above raw tables.
• Renderer: the software that turns a specification or code into pixels and interactive controls, such as Vega-Lite, ECharts, Matplotlib, or a BI product.
• Critic: a model pass that inspects code or a rendered artifact and proposes faults or repairs. A critic is another evidence channel, not an acceptance authority.
• Interaction trajectory: a recorded sequence such as open a tab, change a filter, and inspect the resulting state. A browser can replay it.
• Held-out task: an evaluation task not used as an example while building or prompting the system.
• Ablation: a controlled comparison that removes or changes one component to test whether that component actually caused the observed gain.

State of the field

This synthesis distinguishes repeated findings from environment-specific choices, direct negative or null evidence from merely unsupported claims, and demonstrated capability from open research questions.

Findings in one page

The field is not converging on a single agent architecture. It is converging on a more useful engineering pattern: reduce the part the model must improvise, externalize its decisions, and test the resulting artifact at the layer where failure can occur.

The strongest current techniques are:

1. A typed visualization or dashboard representation between intent and rendering. Raiven, NL2Dashboard, ViviDoc, nvAgent, NL4DV-LLM, current Data Formulator/Flint, DashChat, and the best library-specific skills all constrain generation through a DSL, intermediate representation, or declarative spec. This improves syntax, edit locality, token efficiency, and the ability to validate individual decisions. It does not establish that the underlying question or takeaway is worthwhile.
2. Deterministic work for deterministic claims. Compilers, schema checks, aggregation checks, executable code, browser traversal, replayed interactions, and regression comparisons outperform asking a model to pronounce an artifact correct. The model remains useful for ambiguity, semantic mapping, critique, and repair; it should not substitute for arithmetic or runtime evidence.
3. Grounding in the actual data contract. Governed BI products increasingly bind natural language to semantic models, field descriptions, verified queries, permissions, sample values, and business glossaries. This is more consequential than assigning a generic agent an analyst persona.
4. Mixed-initiative authoring. Data Formulator 2 and DashChat make natural language one control surface among direct manipulation, structured selection, visible transformed tables, history, branching, and reversion. This has stronger human-use evidence than prompt-only generation.
5. Evaluation over the rendered artifact and intended use. DashArena’s new contribution is not another chart score. A system authors an interaction trajectory; a browser replays it; the judge sees task, screenshots, schema, and execution evidence. Human agreement improves materially when interaction evidence is present. This is the closest current answer to the earlier gap around communicative and analytical value, but it is not a measure of reader comprehension, learning, retention, or decision quality.

“Multi-agent” is therefore not a transferable technique by itself. It earns a place only when a role has a different information boundary or tool contract: schema access, transformation execution, visual inspection, browser interaction, or independent acceptance evidence. A planner, composer, and validator can help because they manipulate different representations and evidence, not because their labels simulate a team. Equal-budget single-agent results and several ablation findings still argue against agent count as a default quality lever.

Where the literature converges

The convergence is a common engineering posture, not a standard product architecture:

• Make intent inspectable. Independent systems repeatedly put a visible structure between the request and the render: an analytic specification, dashboard IR, DSL, semantic model, or chart contract. The vocabulary differs; the function is to expose fields, transformations, encodings, interactions, and edits for validation and correction.
• Use deterministic execution for deterministic claims. Recompute values, execute queries, compile specifications, inspect bindings, and replay browser actions. Models can map ambiguous language, criticize, and repair. They should not substitute for arithmetic or runtime evidence.
• Evaluate the delivered artifact. Returned code, structural similarity, and a nonblank render stop too early. Interactive work requires observable state and behavior; explanatory work requires final-context reading and a human gate when comprehension is claimed.
• Ground generation in the real data contract. Governed fields, measures, joins, permissions, verified queries, samples, units, and source vintage are more consequential than an “analyst” persona.
• Keep people able to steer and recover. The strongest authoring evidence favors visible transformed data, direct controls, small edits, history, branches, and reversion over prompt-only regeneration.

Where approaches legitimately diverge

Several competing approaches can each be right in different environments:

• Direct code versus a DSL and compiler. A restricted language can remove coding failures in a stable, specialized grammar; Raiven’s benefit is strongest in scientific visualization. Direct frontier models approach its reported performance in ordinary information visualization, where compiler overhead and an expressiveness ceiling may not pay for themselves.
• Prompt-first versus mixed initiative. Prompt-first creation is proportionate for a simple, reversible one-off. Exploratory or governed work benefits more from direct controls, visible state, history, and correction.
• Open-file tools versus governed BI. Portable code can work across renderers. A native BI assistant can instead inherit maintained semantic models, permissions, and worksheet state. Neither advantage transfers automatically to the other setting.
• One model versus specialized roles. A separate role can help when it owns different tools or independent evidence: schema access, execution, rendered inspection, or browser replay. Dividing one context into persona-labeled roles adds cost without establishing a new capability.
• Static inspection versus interaction trajectories. A static output may be adequately tested through data and final-render evidence. An interactive output needs action coverage and resulting state. A model-authored trajectory is still only a reproducible claim about intended use, not observed user behavior.
• Universal representation versus renderer-specific contracts. A universal IR promises portability; a narrow local contract preserves more expressiveness and is cheaper to maintain. The right boundary is an empirical question.

Direct negative, null, and conditional findings

• DashArena’s deterministic rules agreed with humans only 42.4% of the time, versus 79.8% for its full evidence judge. Removing interaction evidence reduced agreement by 8.1 percentage points.
• No DashArena model exceeded 86% render success or 74% interaction replay. Among 30 cases that executed cleanly but failed overall, 21 contained semantic defects.
• NL2Dashboard found diminishing returns from repeated critic rounds and recommends zero or one round, supporting capped repair rather than indefinite deliberation.
• Removing nvAgent’s processor slightly improved GPT-4o’s average result while harming weaker and multi-table settings. The component is conditionally useful, not universally beneficial.
• Raiven’s advantage was concentrated in scientific visualization; the DSL did not produce the same margin in ordinary information visualization.
• PlotGen’s human review marked 40.5% of outputs completely accurate and another 24.5% somewhat accurate. A sophisticated multimodal loop still produced many materially imperfect plots.
• DashChat users identified mock-data and domain mismatch, acronyms, direct-edit limitations, and styling limits. Rapid prototyping did not erase the need for domain context and correction.

Unsupported shortcuts are a different category

An unsupported claim is not proof of harm. It is simply insufficient evidence for acceptance. The AntV saved “success” results did not include browser render outcomes. The prompt-only packages in the initial sample have no behavioral with/without proof. LIDA’s code-and-text evaluator does not inspect the rendered chart. A validator that fails open is not an acceptance mechanism. Stars, installs, code returned, a nonblank render, synthetic personas, or one vision-model score may be useful inputs or baselines; none establishes data fidelity, integrity, reader understanding, or decision quality.

The model and harness baseline is moving

Shared timeline: September 2023 through August 2026

The three lanes below align general model and harness releases with the changing scope of visualization systems and benchmarks. They are a shared clock, not a causal model: release dates do not prove that a platform milestone caused a paper’s result, and publication dates lag the work. Scores remain comparable only inside the named studies.

The web presentation places every milestone on one proportional September 2023–August 2026 axis. Marker position encodes the date; label width does not encode duration. On narrow screens, the same evidence becomes a vertical chronology inside each explicitly named lane.

By the August 2026 evidence cut, the research question is no longer only whether a model can emit code for a plausible static chart. It is increasingly whether a model plus its harness can preserve meaning, construct a multi-view interactive artifact, exercise it in a browser, and supply evidence that it supports the analytical task. Reliability has not kept pace with that expanding scope.

The baseline has improved materially over the past 36 months, but the published record does not provide a clean visualization-specific learning curve from August 2023 to August 2026. Benchmarks changed, later papers often reran only a subset of models, judges changed, and current systems combine models with different prompts, tools, and test-time budgets. The defensible conclusion is a direction and a set of measured slices—not a universal annual improvement rate.

A fixed scorecard makes the remaining distance explicit

The companion capability scorecard applies one six-level rubric to nine dimensions: task framing; data, semantics, and provenance; construction; interpretation; integrity critique; steering and repair; interaction, responsiveness, and accessibility; reader or decision outcomes; and production, governance, and maintenance. The scale runs from “not demonstrated” through “outcome-proven.” It does not average the dimensions.

On that rubric, the August 2026 field is usable in bounded contexts for data grounding, ordinary construction, interpretation, and mixed-initiative repair; repeatable on bounded tests for integrity critique and delivered interaction; and only demonstrated—not established—for reader outcomes and the full production lifecycle. The same rubric can be applied retrospectively without pretending that unlike benchmark percentages share a common numerical axis. The historical reconstruction shows broad acceleration after 2023, but no common-core dimension yet crossing into field-wide deliverability. Context rows replace the generic ideal with the actual success condition for explanation, exploration, monitoring, decisions, production, scientific work, and reader assistance.

What the closest comparable slices show

The web presentation uses paired points for the before-and-after values below. Each measure keeps its own labeled scale and exact endpoints; readers may compare the two points inside a row, but not horizontal position across metrics. There is deliberately no combined score or common raw axis.

Plot2Code asks a multimodal model to reconstruct a scientific chart as code. An anonymous 2026 preprint re-reported direct, single-pass results on the same Python/Matplotlib subset and normalized scores over the full test set. Between the September 2023 GPT-4V result and the June 2025 Gemini 2.5 Pro result:

The large movement was in visual and textual fidelity, not basic code execution. This is a useful comparison, but it is one preprint’s reconstruction of one chart-to-code benchmark, not a field-wide time series.

CharTide, a peer-reviewed ACL 2026 training study, supplies a second same-paper comparison of GPT-4o and GPT-5 on three chart-to-code benchmarks. GPT-5 improved ChartMimic high-level similarity from 87.7 to 94.7, Plot2Code text match from 52.6 to 61.9, and ChartX’s five-point score from 2.61 to 3.59. Execution moved much less. The pattern is again uneven: generation is becoming more faithful, while harder visual reasoning and semantic alignment still leave substantial headroom. CharTide also shows that a small chart-specialized model can match or exceed a larger general model, so raw frontier scale is not the only route to improvement.

DashArena shows what the 2026 frontier can attempt that earlier chart benchmarks barely measured: a current general model can produce a multi-view interactive dashboard and a replayable intended-use trajectory from an open-ended task. Its top model was competitive with the anonymized human baseline in aggregate preference. Yet no model exceeded 86% render success or 74% replay success, and execution-clean semantic failures remained common. Breadth has expanded faster than reliability.

Dashboard2Code makes another part of dashboard behavior measurable. Its 180 Plotly Dash dashboard-code pairs cover 20 visualization types and eight callback patterns, with 450 interaction tasks. The best reported configuration scored 79.4 overall and 64.2 on the most complex interaction level. DOM access raised one model’s successful UI exploration from 20.61% to 64.58%, showing the value of exposing executable structure rather than relying on pixels alone. The benchmark also exposes a particularly dangerous failure: an interface can respond while hidden state or a transformation is factually wrong. It uses a fixed 1920×1080 viewport and excludes animation and popups, so mobile, responsive, and animated behavior remain outside the result.

Chartography asks a different question: can a model read difficult charts used in professional work? Its 100 practitioner-authored tasks span 12 domain labels and were independently verified by three experts per task. Thirty model configurations were run twenty times per task. The best tested configuration reached 45.0% mean pass@1. Greater reasoning effort helped in eleven of twelve paired comparisons, but the median gain was only 4.5 percentage points and longer reasoning often elaborated an initial visual misread. Because the tasks were deliberately screened for difficulty, 45% is not a prevalence estimate over all professional charts. It is strong evidence that success on basic visual-literacy or chart-QA sets does not imply dependable professional reading.

Scaffolding still helps, but generic instruction is already losing value

The most informative ablations do not say “more prompting is better.” They say that scaffolding helps when it adds a missing representation, tool, or evidence channel.

• In Text2Vis, GPT-4o’s direct pass rate was 26%. Three examples left it at 26%; retrieval plus three examples reached 31%. One structured answer-and-code feedback round reached 42%. Adding visual feedback improved the visual subscores but slightly lowered the final pass rate to 41%. Targeted feedback helped; examples alone were null; another feedback modality was not monotonically better.
• In the original Plot2Code study, adding detailed conditional requirements generally increased resemblance but reduced execution. Gemini Pro’s pass rate fell from 68.2% to 55.3%; the authors found no clear advantage for chain of thought or Plan-and-Solve over the default prompt.
• In nvAgent, removing the processor slightly improved GPT-4o on average while hurting weaker and multi-table cases. This is exactly what a moving baseline should produce: scaffolding that compensates for one generation of model can become redundant or obstructive for another.

The current operational baseline is also no longer a bare model call. Coding harnesses increasingly arrive with planning, repository search, execution, browser inspection, and packaged visualization guidance. A no-skill model is still a useful diagnostic control, but it is not the realistic alternative to a application-specific extension. The decision baseline must be the current model plus the normal harness, its default tools, and any ambient skills or instructions.

A current evaluation-readiness audit shows why that distinction matters. The product-owned historical record now resolves 14 case definitions, 18 run directories, and 13 judge directories, but the current public evaluation surface has only the narrow editorial sentinel. Of the five contexts required for a shared baseline—editorial, exploratory, governed BI, scientific, and interactive—all five now have executable source packets. The exploratory packet uses the exact CC0 Palmer Penguins table to test a real pooled/within-species sign reversal, branch preservation, row accounting, and render receipts. Its widespread use makes it a known-failure sentinel, not held-out performance evidence. The governed-BI packet uses a first-party synthetic semantic model, four roles, six verified queries, four refusals, and a fanout incident that produces $16,830 against the authoritative $7,730. That makes permission and recovery receipts executable, not production security or reliability. The specialized packet uses official 2024 ACS B19013 estimates and margins of error. Iowa, Kansas, Montana, and Wyoming span only $192; their largest internal z-score is 0.149508 against a declared 1.645 threshold. An exact point-only chart can therefore be mechanically valid while its strict winner story is unsupported. The interactive packet uses two fictional Riverbend maintenance snapshots, five canonical state fields, ten exact states, and seven isolated trajectories. Its critical transition makes selected T007 ineligible and leaves only T012 visible; selection, detail, accessible summary, URL, and export must change together. It also contracts history, reset, keyboard, narrow delivery, and a prepared V2 change. Historical auditability and source-packet readiness are not baseline readiness. Do not compare architectures until the exact current model/harness manifest is frozen and bare, normal-harness, and one-added-layer arms can emit equivalent receipts.

The first manifest preflight locked the five packet hashes, local toolchain, available-skill hashes, task order, three arm roles, and one 18-field receipt schema. A credential-free runner now clears the mechanical runner gate: five source checks run before it stages 15 isolated fixture-arm attempts; model inputs are separated from evaluator-only truth; artifact, render, trace, and check files are hashed outside the arm; and receipt identities fail closed. The editorial case required an explicit redaction because its source file contains the answer key. A whole-packet copy would have changed the task.

The successor preflight does not call the comparison ready. Six conditions still block the first call: immutable served-model identity; exportable server- harness configuration; same-model bare access; an authorized capped budget; one versioned Vizier mechanism; and named acceptance authority or explicit not-run treatment. The runner and successor verifier record zero calls and no spend. A passing plumbing self-test, observed gpt-5.6 label, and local codex-cli 0.147.0 remain neither a model result nor a server-side reproduction receipt.

The same finding changes different decisions without changing the evidence:

The interactive packet adds a second audience decision layer:

The manifest preflight changes how those audiences should read comparisons:

The neutral runner turns the same answer into seven concrete decision products:

One layer means one controlled difference

A skill package is an installation and use surface, not automatically a causal unit. The two pinned public Vizier skills combine reader framing, form and pattern retrieval, honesty questions, encoding and palette guidance, deterministic commands, actual-artifact inspection, optional corpus retrieval, and critique composition. Testing either whole package would show the value of that bundle under its budget, not which mechanism mattered.

One candidate is now prepared but deliberately unselected: artifact-evidence reconciliation checkpoint v0.1. It enters once after the first executable artifact and neutral output inventory, before repair or finalization. For each decision-bearing visible claim or state, it records the artifact hash, available source/transform/query/calculation/branch/state evidence, and match, mismatch, or unresolved. Missing states remain missing; discrepancies are frozen before any bounded repair. Overall pass is valid only when every surface matches, no required state is missing, and no repair remains. The one failure target is silent divergence between the delivered artifact and the evidence behind it.

Use this five-part test before calling anything “one layer”:

1. one named mechanism and one expected failure target;
2. one lifecycle insertion point and one versioned output receipt;
3. identical model, harness, inputs, tools, budgets, repair cap, and evaluator;
4. evaluator answers and sentinel mappings kept outside model input; and
5. an explicit owner decision distinct from a passing protocol verifier.

The same candidate changes seven audience decisions without changing its evidence status:

A pass needs the right authority

“Human review” is not one interchangeable box. A context reviewer decides whether an artifact supports the stated job. A domain expert checks semantics and interpretations that deterministic invariants cannot settle. A knowledgeable accessibility evaluator assesses a declared conformance scope. Relevant disabled users show how a delivered task works on their setups. Intended users accept or reject the artifact for the decision. A later independent maintainer supplies handoff and change evidence.

These roles cannot be filled by an automated critic. The W3C evaluation overview says no tool alone determines accessibility. Its guidance on involving users also says user evaluation complements rather than replaces standards work and must report participant scope without overgeneralizing.

The prepared E0 candidate makes that boundary executable:

Five fixtures times six lanes yields 30 explicit not-run records. A scope receipt rejects a human pass without a named human authority and rejects an automated tool as that authority. A deterministic result may therefore be reported only as a mechanical pass for the named fixture and arm—not as unqualified acceptance, accessibility, usefulness, comprehension, maintenance, or Vizier efficacy. The research owner has not selected mechanical-only scope or named people, so this is a candidate policy with zero model calls and six remaining preflight blockers.

The audience consequence is concrete:

Six answers, then one release

The six remaining E0 blockers are now one decision register, not one blanket approval. It offers 17 admissible paths across four authority roles and keeps each answer separate:

An unavailable surface, declined budget, requested revision, or rejection is a valid recorded decision, but it cannot make the run ready. Even six advancing answers do not trigger a call: the research owner must separately release the hash of that resolved decision set against a named run plan, call cap, and stop condition. Decision receipts may identify a non-secret route and hash evidence; credential values never enter custody.

The verified candidate rejects wrong-role, incomplete, secret-bearing, partial, and six-answer-without-release states. That is protocol integrity, not approval. All six selections and the final release remain null, spend and readiness remain false, no person was contacted, and no model was called.

The audience translations make the same state useful without changing it:

Score layers, not “quality”

The prepared E0 scorer has no overall score, winner, rank, universal pass, or efficacy field. It reports two mechanical tiers—computable checks and checkable obligations—plus six named nonmechanical lanes: context review, domain review, accessibility conformance, disabled-user evaluation, intended-user acceptance, and maintenance handoff. Cost stays beside those outcomes.

A mechanical tier passes only when it contains at least one check and all checks pass. One failure yields fail; an empty tier or any not-run yields incomplete. Each nonmechanical result retains its exact authority, rationale, scope, and evidence. not-run remains missing data, and one human role cannot sign another role’s lane.

The zero-call verifier created 15 synthetic fixture-arm ledgers and preserved 90 nonmechanical not-run records. It rejected unnamed and cross-lane human authority and injected aggregate score/pass fields. That establishes scoring plumbing, not 15 successful artifacts or a Vizier effect.

Integrity and readability need a blinded test

R4 now has a preregistered pilot and five frozen inputs, not a result. Five opaque-ID SVGs, five neutral briefs, and nine source-evidence files make one 19-file model-input allowlist. Five context mappings and known-defect records stay in separately hashed evaluator-only custody. The model can inspect the same visual and evidence under either lens without receiving the answer key.

The two lenses now also have one hashed prompt and one fail-closed raw-response schema each. A response may complete with evidence-grounded findings, complete with no findings, or explicitly abstain. Twelve synthetic adverse records show that aggregate fields, mixed lenses, wrong identities, incoherent evidence, and findings attached to an abstention are rejected. This is tested local plumbing; it does not show that a provider supports the format or a model will follow it.

Three exact-model slots, the independent human data editor, calibrated tie rule, budget, release, and results are still empty. Future models produce separately blinded integrity and readability observations without scoring themselves; the editor later scores them against hidden defect truth and a declared readability rubric. Only the two within-lens partial orderings are compared. They never become one quality score.

The remaining approval path is now explicit. Five gate categories become seven attributable receipts: one exact served receipt for each of M1, M2, and M3; editor-signed role acceptance; same-editor calibration; a research-owner capped budget; and a final research-owner release over the exact six- prerequisite digest. The budget must cover 30 primary records—three models by five stimuli by two lenses—with retries capped separately.

Twenty advancing and non-advancing receipt paths validate. Sixteen synthetic adverse states show that wrong authority, cross-subject choices, incomplete or unsafe evidence, duplicate/partial prerequisites, a refusal, missing release, wrong digests or hashes, different editors, route mismatch, call-cap mismatch, and duplicate model identity remain blocked. This tests the handoff mechanism; all six real prerequisites are pending and release is false.

The editor’s future calibration task is now prepared but blank. It contains ten baselines—five stimuli under each lens—and ten synthetic response-quality vignettes, including two A/B-swapped repeats. The response can complete, request revision, or state that usable resolution cannot be established; fourteen adverse custody, coverage, repeat, state, and aggregate-result records fail. No editor, appointment, calibration, resolution, or tie release exists.

The same register changes the decision question by audience. Practitioners can ask whether all seven receipts are inspectable. Analytics leaders can use them as a pilot-control sheet. Researchers can retain refusals and provider failures as results about the protocol rather than silently changing it. Newsrooms keep provider, adjudication, research release, and publication authority separate. Product teams can re-hash a route adaptation instead of hiding it in the runner. Education and accessibility readers can see that editor calibration is not learner or disabled-user evidence. Broad readers get the shortest honest status: the approval path is testable, but every real answer is blank.

For practitioners and analytics leaders, the additional check is whether the editor wrote the baseline before seeing model output. Researchers should treat the swapped repeats as a consistency check, not inter-rater reliability. Newsrooms retain editorial/publication authority; product teams keep the packet evaluator-only; education and accessibility readers should not read review- scale calibration as comprehension or disabled-user evidence. Broad readers can say only that the editor now has a concrete future task.

What can and cannot be projected

The observed direction supports three bounded forecasts:

1. Likely to commoditize: syntactically valid chart code, conventional chart selection, basic styling, routine repair, and generic “inspect your render” advice. These should be short, replaceable defaults, not a large permanent application-specific doctrine.
2. Likely to migrate into models or harnesses: generic planning, self-review, browser/tool use, and library navigation. A visualization system should consume these when they work and avoid duplicating their orchestration.
3. Unlikely to be solved by baseline capability alone: the local question, audience, semantic definitions, authoritative data, denominator, source custody, consequence, publication boundary, environment-specific evidence, and real-reader outcomes. These are facts and authorities the model does not acquire merely by becoming better at code or vision.

Linear projection would be false precision. The practical forecast is that the value of generic instruction will decay fastest, the value of executable local evidence will persist, and the value of context and authority boundaries will increase as agents become capable of taking more consequential action.

How evidence was graded

The same word—“works”—covers incompatible outcomes in this literature. This report keeps seven layers separate:

Passing a lower layer is a prerequisite, not a proxy for all higher layers. “Rendered,” “valid,” “high similarity,” and “preferred by a vision model” are different claims.

Evidence labels used below:

• A — comparative human/outcome evidence: a relevant controlled user or human-calibrated task study, with material caveats still reported.
• B — comparative benchmark evidence: a held-out or public task set with explicit metrics and baselines, but not a direct reader outcome.
• C — executable mechanism evidence: inspectable implementation, tests, or evaluation fixtures without persuasive independent efficacy evidence.
• D — product or author claim: first-party description, demo, popularity, or anecdote without a usable comparative outcome.

Technique map

Research and implementation landscape

This landscape shows where research techniques appear in working systems, agent-skill packages, and governed BI products. The entries are examples and evidence records, not product scores: a paper can support a bounded performance claim, a repository can expose an implementation, and product documentation can describe a contract, but those are different kinds of evidence.

Full tools and research systems

Skill and plugin packages

These packages make a useful maturity ladder:

static advice
  -> versioned constraints and on-demand retrieval
  -> explicit design/build artifacts
  -> deterministic source/code checks
  -> final renderer/browser inspection
  -> interaction coverage and evidence receipts
  -> held-out behavioral comparison with and without the package
  -> human outcome study

SciVisAgentSkills reaches the paired-ablation rung for specialized scientific work, but not independent reproduction or a human outcome. Vizro reaches furthest on execution evidence; AntV has the largest included retrieval/code benchmark; the OpenAI skill has the broadest chart-contract and delivery QA. Those are different strengths and should not be collapsed into an install count or one “best skill.” The companion skill-package deep dive compares 18 files or families and separates registry installations from repository popularity and behavioral evidence.

Governed commercial environments

The commercial systems are especially environment-dependent. Their main advantage is not a universally better model. It is custody of semantic models, permissions, field metadata, verified queries, authoring state, and the native renderer. That advantage does not transfer to an open-file editorial workflow unless another system is given equivalent data contracts.

The evaluation frontier

DashArena materially changes the map

DashArena, published 2026-08-11, contains 234 open-ended tasks derived from high-quality Tableau Public dashboards across 14 clusters. A candidate returns a single-file ECharts dashboard and a structured two-turn interaction trajectory. A Playwright executor replays the trajectory and gives the pairwise judge task context, screenshots, schema, and execution evidence.

The strongest results are about the evaluation method:

• Six dashboard-experienced human annotators labeled 100 pairs, three labels per pair; 99 were evaluable. Human inter-rater agreement was only moderate (Fleiss kappa .384), and 52 pairs were unanimous. Dashboard quality is not a naturally objective scalar.
• The distilled DashJudge-8B agreed with humans 79.8% of the time (kappa .600). Removing interaction evidence reduced agreement to 71.7% (kappa .441), while deterministic rules alone achieved 42.4% (kappa .095). Intended-use evidence added 8.1 percentage points.
• A 100-trajectory audit found 99.8% valid targets and 96% all-pages coverage, but per-page control coverage was 80.8%. Of 841 actions, 97.3% were attributable and 93.3% matched the intended behavior; the other 6.7% exposed errors rather than hiding them.
• On 120 held-out tasks, the top model, GPT-5.5, scored 1449 versus a human baseline at 1321, with overlapping uncertainty. The authors explicitly treat the human dashboard as a practical anchor, not an oracle, because it predates the derived task.
• No model exceeded 86% render success or 74% trajectory replay success. In a targeted 100-failure audit, construction/runtime caused 26.9%, interaction 25.4%, data computation/binding 22.4%, presentation/readability 16.4%, and analytical issues 9%. Of 30 execution-clean failures, 21 still had semantic defects.

This qualifies the earlier conclusion that communicative-value evaluation was empty. DashArena now provides a serious, human-calibrated measure of task-grounded analytical support and interaction quality. It does not reverse the conclusion about readers. The benchmark does not observe whether a target audience comprehends an argument, learns a concept, remembers the message, or makes a better real decision. Its pairwise aggregate is also not a universal taste or audience model. The right update is “a major middle layer now exists,” not “communicative value is solved.”

The naming has outrun the instrument

Two further 2026 benchmarks reach toward communicative value and then measure something else. Reading them together is more informative than reading either alone, because they fail in the same direction from opposite starting points.

SciVisAgentBench is the most systematic scientific-visualization agent benchmark released so far: 108 expert-crafted cases over a four-dimensional taxonomy, a multimodal evaluation pipeline, and an IRB-approved validity study with 12 domain experts. Its taxonomy makes Scientific Insight Derivation a first-class visualization operation. But that is a task the agent performs, not a property of the artifact that is scored. Scoring is rubric agreement with a single expert-authored ground-truth visualization, judged 0–10 by an MLLM, plus PSNR, SSIM, and LPIPS similarity to that reference, plus token and time efficiency. Tasks are deliberately curated to admit one explicit outcome. The 12 experts are not readers being measured; they are raters whose agreement with the LLM judge is being measured. The paper is candid that open-ended goals with multiple valid outcomes “remain difficult to evaluate reproducibly at a benchmark scale,” and names the same obstacle from the inside: different visualizations may convey the same insight, which complicates automated scoring.

MultiVis-Agent goes further in language and no further in instrument. Its evaluation has an explicit high-level perceptual layer of six weighted dimensions — chart-type appropriateness, spatial layout, textual elements, data representation, visual styling, global clarity — described as “capturing human-perceived effectiveness.” The scorer is a vision-language model applying rubrics to a rendered image. No human reader appears anywhere in the evaluation.

The gap, stated precisely, is no longer that the field has failed to name communicative value. Both of these name it, or something adjacent to it. The gap is that naming has outrun measurement, and a model judge scoring a rubric is increasingly being asked to stand in for a reader. That substitution is convenient, reproducible, and cheap, and it is not the same evidence. A benchmark can report a high perceptual score for a chart that no human has read.

For anyone selecting a benchmark: read what the metric is computed on, not what the metric is called. A dimension named for clarity, effectiveness, or insight may be a model’s rubric score against a reference image. That is a useful signal about conformance and a weak one about communication.

Dashboard2Code exposes state that a screenshot can hide

Dashboard2Code reconstructs interactive Plotly Dash applications from screenshots, optional DOM, and interaction. Its benchmark combines 58 real-world seed dashboards with 122 generated examples, then checks visual fidelity, code, dynamic browser behavior, and 450 interaction tasks. Ninety generated dashboards were also scored by three visualization- experienced graduate evaluators; the final automatic metric correlated .781 with their ratings.

The most important result is not one total score. DOM access sharply improves exploration, complex callbacks remain harder, and systems can produce a visually responsive but factually wrong state. A static image judge cannot see that failure. The scope boundary matters just as much: Plotly Dash only, fixed desktop viewport, no animation or popups, and no reader study.

Chartography makes professional reading a separate capability

Chartography, released 2026-08-11, contains 100 difficult professional chart-reading tasks across 12 domain labels. Each task was authored by a practitioner and independently verified by three experts, including an acceptable answer range. The best of 30 frontier configurations reached 45.0% mean pass@1 over twenty trials per task.

Higher reasoning effort usually helped, but not enough to erase the main failure: sparse axes, 3D projections, contours, and domain conventions can be misread before reasoning begins. More tokens then produce a longer explanation of the wrong visual premise. The benchmark is adversarially difficulty- screened, so it does not say models fail on 55% of ordinary charts. It does say that a general “chart literacy” score is too broad a release gate for professional, consequential reading.

Raiven narrows generation failure in a particular environment

Raiven’s .988 VMPC is strong counterevidence to treating direct-code chart generation failure rates as immutable. A formal DSL plus compiler can nearly eliminate many specified-mark, encoding, linking, hallucination, and execution failures. But the benchmark fully specifies the target visualization, and the largest delta occurs in 3D/scientific tasks where generic web code generation is weak. In ordinary 2D information visualization, direct frontier models nearly match Raiven on its own metric.

The transferable claim is conditional: when an environment has a stable visual grammar, recurring hard syntax, and deterministic rendering, invest in the representation/compiler. It is not evidence that a DSL can choose the right question, identify a misleading comparison, or replace final human judgment.

Benchmark construction is part of the technique

The companion benchmark crosswalk documents the input object, requested output, source distribution, grader, human context, supported claim, and nonclaim for the benchmarks carrying the main conclusions in this review. It reveals a measurement progression:

1. exact or near-exact code/spec matching;
2. syntactic legality and render success;
3. data/encoding correctness against a reference;
4. chart extraction, visual grounding, question answering, and comparison;
5. integrity or quality judgment with human calibration;
6. interaction state and task-grounded replay;
7. actual user performance, comprehension, or decision outcome.

Many impressive percentages in tool repositories live at levels 1–2. nvAgent and Raiven reach level 3 in bounded ways. ChartQAPro, Chartography, ChartDiff, and the two 2026 PolyChartQA benchmarks expand level 4 across realistic, professional, comparative, multi-chart, and multilingual cases. Misviz, VisJudge, and misleading-chart robustness studies measure parts of level 5. Dashboard2Code and DashArena reach level 6. Data Formulator 2 and DashChat provide small, contextual evidence at level 7 for authoring/prototyping experience, not for reader comprehension or decision quality. No one result spans the stack.

Three construction effects now deserve the same attention as model scores. Human-authored multi-chart questions were up to 27.4 percentage points harder than model-generated questions; ChartDiff’s lexical-overlap metrics favored systems that its human-aligned judge rated much worse; and misleading-chart system rankings changed between controlled synthetic and heterogeneous real charts. A benchmark is a task, distribution, and grader—not a neutral name.

Five validity threats block a configuration verdict

The latest audit asks a narrower question than the landscape: does this evidence authorize one Vizier configuration? The answer is no. It records five distinct boundaries rather than hiding them in one confidence label:

Two direct human/model comparisons make the benchmark boundary concrete. CHART-6 evaluates eight models on 851 items from six visualization-literacy assessments, ten times each. The tested models underperformed the held human data on average, and every item-level model error pattern remained below the human noise ceiling. A newer high-level interpretation study compares 24 people’s descriptions with three multimodal models over 60 charts. Models matched the study’s bounded analytical designer intent more often, yet tended to enumerate structure and values where people more often built narratives and used visual scaffolding.

Those findings are not inconsistent once task, model, prompt, scoring, and vintage remain attached. They show why aggregate task success, human-like errors, analytical-intent matching, and reader experience are separate claims. The public rule is therefore strict: a benchmark licenses only its named task-and-grader lane. Overall quality, audience fit, comprehension, accessibility, accepted delivery, maintenance, and a configuration winner each require new evidence.

Research gap ledger

An open question is not the same as an empty field. Some gaps now contain a bounded study, some contain evidence that a failure occurs, and some still lack the measurement needed to make a decision. The status language below is literal:

• Measured but bounded means a public comparison exists but does not yet generalize to ordinary production.
• Failure demonstrated means research establishes that the failure occurs, not its population frequency or best remedy.
• Partly answered means a human study covers one task, audience, or time window while leaving consequential transfer questions open.
• Open means the reviewed corpus does not contain direct outcome evidence for the question as posed.

This ledger changes the research agenda in two ways. First, it prevents a single new paper from being narrated as “the gap is solved”: the relevant row moves only as far as the evidence permits. Second, it makes research pursuits testable. A skill-package survey must eventually lead to an ablation, not a popularity ranking. A specialized-vision survey must identify a same-task, equal-budget critic comparison, not merely another chart-question-answering leaderboard.

Active refresh triggers

The following releases or studies would materially change one or more rows:

• DashArena releasing its task set, executor, and judge for independent reproduction, or adding coding-agent, non-Tableau, numeric-fidelity, and real user-outcome tracks;
• Raiven releasing code and third parties reproducing its scientific- visualization results with independent graders and exploratory tasks;
• a public equal-budget visualization study isolating role/tool decomposition from extra inference;
• independent current-model evaluations of generic, dashboard, accessibility, mobile, and explanatory-visualization skills against the normal harness;
• same-task, equal-budget routing comparisons among deterministic checks, current general vision, and narrow chart specialists on end-to-end critique;
• a Graphy workshop-to-commit and model receipt, or a versioned Graphy release with formal or in-situ representative recheck after a material change;
• repair of the unidentified Education row in the 2025 decision-visualization review’s supplement, followed by a stable-ID row audit against GenAI role, immutable artifact, delivery, decision or calibrated-trust outcome, and later-recheck fields;
• reader studies measuring comprehension, retention, trust calibration, accessibility, mobile use, or consequential decisions from generated visualizations;
• current Data Formulator studies covering governed sources, persistent agent threads, and free exploration rather than reproduction;
• commercial BI vendors publishing reproducible error rates or user outcomes stratified by semantic-model quality and task type; and
• evidence that small, local deterministic contracts fail to transfer across the renderer and publication environments a system actually serves.

Sources inspected

Pinned repositories

• Microsoft Data Formulator at 5d4f7b3, MIT.
• Microsoft LIDA at d892e20, MIT.
• HoloViz Lumen at 7d587f3, BSD-3-Clause.
• McKinsey Vizro at b1d7b11, Apache-2.0.
• AntV chart visualization skills at b47f2fe, MIT.
• Markdown Viewer skills at a3afd45; no repository license identified in the captured metadata.
• OpenAI role-specific plugins at fe5608d, MIT.
• NTCoding claude-skillz at 21c6101; no repository license identified in the captured metadata.

Papers read in full for this update

• Wang and Deng, DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation, 2026.
• Irger et al., Raiven: LLM-Based Visualization Authoring via Domain-Specific Language Mediation, 2026.
• Shi et al., NL2Dashboard: A Lightweight and Controllable Framework for Generating Dashboards with LLMs, 2026.
• Ouyang et al., nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow, ACL 2025.
• Shen et al., DashChat: Interactive Authoring of Industrial Dashboard Design Prototypes through Conversation with LLM-Powered Agents, 2025.
• Wang et al., Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way, CHI 2025.
• Sah et al., Generating Analytic Specifications for Data Visualization from Natural Language Queries using Large Language Models, 2024.
• Goswami et al., PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback, 2025.

Human-outcome studies added by the gap-ledger pass

• Kuo et al., Vibe Visualizing: Exploring the Human-AI Interaction Dynamics in AI-Assisted Visualization, 2026 preprint: 20 novices, 60 sessions, and 175 charts.
• Kwon et al., Visualizationary: Longitudinal Critique of Visualization Design, 2024 preprint: 13 designers working on self-selected visualizations over three to five days, plus three expert raters.
• Yan et al., The Effects of Generative AI Agents and Scaffolding on Visual Analytics Comprehension, 2024 preprint: randomized 117-person comparison of data stories, passive GenAI, and proactive scaffolded GenAI.
• Palani and Setlur, Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics, CHI 2026: 22 developer interviews, 16 professional end-user observations, and a two-week field deployment with six developers, 38 experiments, 57 newly authored cases, ten models, and six system prompts.
• Ardito et al., AI-Supported End-User Development for Data Visualization, 2025: one-company exploratory study with eight interviews and three design-probe sessions.

Benchmark and training studies for the moving-baseline analysis

• Verma et al., CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models, 2025: eight models, 851 common items, six human-designed assessments, and item-level human error-pattern ceilings.
• Jeon et al., How Do LLMs See Charts? A Comparative Study on High-Level Visualization Comprehension in Humans and LLMs, 2026: 60 charts, 24 human participants, three multimodal models, and three prompt constraints.
• Wu et al., Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots, Findings of NAACL 2025.
• Rahman et al., Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text, EMNLP 2025.
• Zheng et al., CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution, ACL 2026.
• RRVF: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback, anonymous ICLR 2026 submission. Used only for its clearly labeled same-benchmark historical table and treated as preprint evidence.
• Li et al., Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards, 2026 preprint: 180 dashboards, 450 interaction tasks, and a 90-dashboard human validation of the evaluator.
• Chartography: A Benchmark for Professional Chart Understanding, 2026 preprint: 100 practitioner-authored, three-expert-verified tasks and 30 model configurations.
• Xu et al., POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering, ACL 2026: 22,606 charts and 26,151 QA pairs across ten languages.
• Ymyang et al., Chart-MRAG: Benchmarking Multimodal Retrieval Augmented Generation on Chart-based Documents, ACL 2026: 4,738 expert-validated QA pairs across chart-bearing real-world documents.
• Ye, ChartDiff: A Large-Scale Benchmark for Comprehending Pairs of Charts, ALVR 2026: 8,541 chart pairs and a direct lexical-versus-human-aligned metric comparison.
• Efat et al., Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts, Findings of ACL 2026: 534 multi-chart figures and separate human- and model-authored questions.
• Shu et al., FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models, ACL 2026: 1,200 real financial charts, 7,016 questions, and 26 evaluated models.
• Tonglet et al., Protecting multimodal large language models against misleading visualizations, ACL 2026: controlled vulnerability and six-intervention study over misleading and non-misleading charts.
• Laskar et al., Lost in Translation: Do LVLM Judges Generalize Across Languages?, Findings of ACL 2026: multilingual multimodal judge evaluation with a chart-centric subset.

Reader and accessibility study added by the open-question pass

• Touching or Chatting: LLMs and Tactile Charts for Blind and Low-Vision Chart Learning, 2026 preprint: 12 participants and 263 substantive queries; strong multimodal preference and spatial-model benefit without measured accuracy improvement.

Foundational task and environment frameworks

• Brehmer and Munzner, A Multi-Level Typology of Abstract Visualization Tasks, IEEE TVCG / InfoVis 2013.
• Segel and Heer, Narrative Visualization: Telling Stories with Data, IEEE TVCG / InfoVis 2010.
• Sarikaya et al., What Do We Talk About When We Talk About Dashboards?, IEEE TVCG / InfoVis 2018.

Current first-party product documentation

• Power BI: use Copilot with reports and semantic models, last updated 2026-05-26 when inspected.
• Tableau Agent FAQ, including July 2026 dashboard beta and current limitations.
• Looker Conversational Analytics overview, including LookML grounding, data agents, verified queries, and Advanced Analytics.

Model and harness milestone records

• GPT-4V system card, 2023-09-25.
• GPT-4o announcement, 2024-05-13.
• Claude 3.5 computer use beta, 2024-10-22.
• Claude Code research preview, 2025-02-24.
• Responses API and agent tools, 2025-03-11.
• GPT-5 for developers, 2025-08-07.
• Hosted computer environment and reusable agent skills, 2026.

Earlier evidence retained in the comparison

The underlying research packet also contains VisEval, MAST, Cleveland–McGill MLLM testing, Does It Run, VIS-Shepherd, Text2Vis, CoDA, RealChart2Code, equal-budget single-agent results, synthetic-persona studies, and LLM visualization literacy work. Those sources continue to control the claims about reader simulation, integrity versus readability, equal-budget coordination, and benchmark coverage. This update does not silently promote abstract-only captures from that packet to full-text evidence.

Update log

• 2026-08-18 — Named the gap more precisely: the naming has outrun the instrument. Two 2026 benchmarks that had been seen by title only were read in full. SciVisAgentBench makes “Scientific Insight Derivation” a task in its taxonomy but scores rubric agreement with one ground-truth image plus PSNR/SSIM/LPIPS, and its 12-expert study measures human–LLM judge agreement, not comprehension. MultiVis-Agent describes a perceptual layer “capturing human-perceived effectiveness” and instruments it with a VLM scoring six rubric dimensions, with no human reader in the evaluation. The standing conclusion is unchanged but sharper: the field now names communicative value and measures conformance, and a model judge is being asked to stand in for a reader. Evidence cut moved to 2026-08-18.

• 2026-08-16 — Retired a repeated retrieval route without promoting an access gap. Three bounded B91 passes now support a named-surface stop rule: the 6 / 20 / 1 / 0 content ledger is unchanged, one row is access-blocked and unassessed, zero generic B91 targets remain active, and four exact conditions can reopen it.

• 2026-08-16 — Mapped InfoViP research contacts without inventing operating authority. The closed Elsa/API/UI opportunity names three research mentors but supplies no execution receipt; its nonemployee boundary cannot furnish governmental approval. An adjacent AI-QA project does not name InfoViP or a completed audit. Operation, maintenance, QA execution, release approval, authorization, and the application join remain unassigned.

• 2026-08-16 — Repaired InfoViP’s chronology and application boundary. A future-tense FDA page is explicitly content-current to 2024, a 2026 FDA biography names a project lead without operations authority, and the Elsa 4.0/HALO launch contains no InfoViP join. Search recency and agency-level release no longer distort the maintained episode.

• 2026-08-16 — Final InfoViP report adds a production component and its QA afterlife. Added processing approval, AWS/AERS integration, >30 million historical plus ~8,000 daily submissions, bounded routine case-series use, and monitoring. Kept the absent QA plan, planned audits, incomplete roles, uninvestigated downstream effect, current no-ATO record, and all complete episodes visible. A 2026 Elsa/API/UI opportunity is prospective.

• 2026-08-16 — InfoViP gained counted field-use and award-envelope receipts. Added 20 unique reviewers and 58 submissions from a six-month internal real-work evaluation, a later 29-million-history plus daily operating pipeline, and three FDA awards totaling $2.40 million obligated / $2.86 million estimated. Kept acceptance/ATO, current routine use, outcome, affected-reviewer return, whole cost, ROI, and all complete episodes absent.

• 2026-08-16 — InfoViP added an operational-pipeline and governance- counterevidence row. Joined co-design, an operational pilot, 28-million- report plus incoming processing, and a later performance report while the current inventory still says development, implementation N/A, and no ATO. The exact-artifact audit is now 15 named cases and zero complete episodes.

• 2026-08-16 — One exact publisher abstract reduced the unassessed ledger to one. B92 adds a bounded non-AI multicriteria, Monte Carlo, and rank-stability method at the abstract layer. B91 remains content-unassessed; the 27-row ledger is 6 full / 20 abstract or official / 1 unassessed / 0 complete AI lifecycles.

• 2026-08-16 — Partial source lineage kept below a deployed build. The live DiscoverWater page differs from the sole published v1.2 page state, while eight of twelve dependency names and two of three canonical asset samples support partial shared lineage. The prototype demonstration and analytics hooks add no accepted-build, use, outcome, accessibility, cost, or AI receipt.

• 2026-08-16 — Final-three recovery found one public-delivery afterlife. B110 now has an official abstract, pinned framework and application source, and a same-named DiscoverWater interface reachable at KU. At that pass B92 and B91 remained content-unassessed. The 27-row ledger was then six full texts, 19 abstract or official-content rows, and two unassessed; no exact live-build join, use denominator, audience outcome, recheck, or AI lifecycle was admitted.

• 2026-08-16 — Residual DOI content recovery stopped at three unknowns. All 14 residual rows were investigated; eleven gained substantive primary content, including one full chapter, while three remain metadata-only. B102 adds an evaluation-to-redesign mechanism and B78/B38 add practice context, but no explicit GenAI role, accepted release, routine-use denominator, consequential outcome, or later affected-audience recheck.

• 2026-08-16 — Seven human evaluations kept below one deployed lifecycle. Nine high-signal DOI rows yield eight primary abstract or official-project surfaces and seven explicit participant/evaluator studies. InfoViP adds one NLP and unsupervised-learning production-intent near-miss; future-tense installation remains below accepted release, routine use, outcome, and later recheck. Fourteen DOI rows still lack substantive primary content.

• 2026-08-16 — Non-DOI recovery repaired authority before classifying evidence. Eight stable keys yielded three DOI recoveries, two year corrections, one rejected foreign PMID, four full texts, and one gated row. The strongest receipt is a non-AI cancer-diary embedded-use/usability comparator; zero complete AI lifecycles were added.

• 2026-08-16 — Human studies kept below deployment. Primary-text recovery for A208’s 27 no-abstract DOI rows puts five exact texts into custody: four empirical papers and one review. Controlled audience measurement, public- sector co-design/demo, and enterprise production intent remain three artifact-specific receipt lanes; none reaches accepted field release plus later affected-audience recheck. Twenty-two DOI rows remain unassessed.

• 2026-08-16 — Mature non-AI afterlife kept out of the AI lifecycle. Exact- DOI discovery covers 114 stable keys and 87 abstracts. AIDSVu is the sole five-category lifecycle signal, adding ten years of delivery, aggregate use, governance, and a later data release—but no AI role or affected-audience recheck. The remaining abstract and identifier gaps stay unclassified.

• 2026-08-16 — Artifact custody separated from audience delivery. A deterministic title-cue screen over 122 authoritative A208 keys surfaces one explicit GPT course study and one pinned 1,271-blob supplement. It yields zero audience-delivery or later-recheck chains; 121 title nonmatches remain not surfaced rather than excluded, and the lifecycle denominator stays null.

• 2026-08-16 — Longitudinal evaluation fills the development-decision middle, not the lifecycle. Lexara adds six developers using a deployed CVA evaluation toolkit for two weeks across 38 experiments, 57 newly authored cases, ten models, and six prompts. The combined twelve-receipt audit covers 14 named cases and still finds zero exact artifact carried through accepted delivery, consequential audience decision or calibrated trust, whole cost, and post-release same-lineage recheck.

• 2026-08-16 — Full six-domain crosswalk isolates five authority gaps. The 127 supplement positions now map to 122 distinct A208 review keys. Five exact-looking DOI candidates remain outside the review-controlled frame, one printed PMID resolves to an unrelated paper, and lifecycle classification remains unstarted and null.

• 2026-08-16 — First domain crosswalk finds a second authority gap. Eleven printed Education labels plus the B59 overlay resolve to twelve distinct A208 bibliography/DOI keys. M. Lu (2020) has no A208 bibliography record; a plausible Crossref candidate stays outside the authoritative frame. The Education crosswalk remains 12 of 13 and lifecycle classification remains null.

• 2026-08-16 — Education placeholder repaired; stable-key audit still open. A208’s post-selection Education paragraph and B59 bibliography uniquely reconstruct the published ? as Hernández-Calderón et al. (2023), DOI 10.1093/iwc/iwac043. The gap ledger retains the broken source value, distinguishes the overlay from a publisher correction, and keeps lifecycle classification null until all rows have stable keys.

• 2026-08-16 — Review inventory repair placed before lifecycle screening. A208’s supplement preserves domain totals of 127 but identifies 126 citations and one literal question-mark Education row. The gap ledger now keeps count, identity, and lifecycle classification separate; no placeholder becomes a negative finding.

• 2026-08-16 — A review corpus is not a lifecycle denominator. Added a 127-study decision-visualization review as a broad coverage map. Because the review does not specifically focus on GenAI and does not jointly extract the eight same-artifact receipts, the qualifying lifecycle count remains null, not zero; primary-paper classification is the next denominator-changing step.

• 2026-08-16 — Same endpoint, different retry burden, no cost winner. The VisCoder2 audit reconstructs 584 versus 714 conditional revisions over 888 tasks before both routes reach 732 execution passes. Missing comparable debug-resource telemetry and human or production acceptance keep the result at attempt topology; the total-cost ledger is now v7 with 0/12 accepted-cost comparisons.

• 2026-08-16 — Production recovery states separated by actor. Three held dashboard afterlives now resolve to two maintainer-verified restorations, zero affected-actor recoveries, zero authority transfers, and zero complete twelve-state rows; the gap ledger names the first valid upgrade receipt.

• 2026-08-15 — Reader outcomes split by AI role and lineage. A bounded ten-row audit now separates one direct controlled AI-created-chart reader effect, five adjacent human-outcome rows, three creator/co-design rows, and one model-reader-only row. Zero reaches accepted delivery plus later same- lineage reader recheck.

• 2026-08-15 — Governance split into two evidence rails. Three provider control contracts and two named-feature organizational-use reports are now shown as real but non-substitutable evidence, alongside two adjacent governance-process studies. Zero of seven held rows clears the nine-receipt same-deployment join.

• 2026-08-15 — Acquisition separated from adoption. Twelve named skill listings have public install signals, but collapse to ten parent repositories and eleven documented lineages. The seven-stage public ladder now keeps listing and acquisition separate from successful presence, invocation, retention, organizational acceptance, and outcomes; 0/12 named rows reaches the latter behavioral stages in held public evidence.

• 2026-08-15 — Eleven-stage system anatomy added. Six primary authoring or analysis systems all separate generation, execution, and some critique route; four expose meaningful human control; zero reaches accepted delivery, intended-reader outcome, or later maintenance. The public map assigns information and authority without selecting one universal agent topology.
• 2026-08-15 — The capability-to-practice bridge is short, not empty. Ten primary cases yield three controlled partial joins—to analyst use, analyst correction, and reader harm—but zero exact-version accepted-delivery bridge and zero complete episode. The gap ledger retains the first missing receipt rather than composing a lifecycle across studies.
• 2026-08-15 — Fixed-budget near miss added. Selective TTS matches declared LLM-call and separate output-token budgets inside one visual-insights pipeline, but its released accounting omits repair and errored-worker cost and its candidates are not contract-accepted artifacts. The maintenance and cost row now distinguishes declared partial budget, observed route-wide use, and accepted-output cost; the complete result is 0/11.
• 2026-08-15 — Six-receipt lifecycle null made explicit. Seven leading cases were audited across repeated representative use, immutable tested build, exact model, versioned release, later event, and post-change recheck. Zero clears the full join; the null is search-scoped and retains every partial lane and reopen condition.
• 2026-08-15 — Longitudinal co-design added without lifecycle promotion. Graphy follows three blind co-designers through 12 sessions, four workshops, eight months, and implemented interaction changes. Its paper disclaims formal evaluation, and its seven-commit public repository has no workshop binding, release, or representative recheck; zero new complete episodes were added.
• 2026-08-15 — Accepted-artifact denominator added. ChartAgent’s accuracy/tool-call sweep adds one partial cost proxy, but zero of nine held fragments has equivalent route cost plus frozen accepted outcomes. Ledger v5 retains failures in the numerator and rejects benchmark accuracy, tool calls, and per-attempt averages as accepted-cost substitutes.
• 2026-08-15 — Study surface found; tested build still unknown. MAIDR’s public legacy code now supplies a dedicated AI-study surface, a v2.10.0 version floor, and real later maintenance. The abstract-level eight-person result has no exact tested-build or model join, the later changes have no representative recheck, and the TypeScript rewrite is a new validation target.
• 2026-08-15 — Four near-misses, zero new lifecycle episodes. Added a seven-key horizontal admission rule, representative pre-AI MAIDR accessibility evidence, the current AI rewrite boundary, and a described-but-removed human study. Useful evidence stays in its lane; project history does not become an artifact afterlife by aggregation.
• 2026-08-15 — Five validity threats block configuration promotion. Added CHART-6 and a high-level human/model interpretation study, separated task success from human-like errors and reader experience, forbade benchmark lift, and kept the qualifying current Vizier configuration-run count at zero.
• 2026-08-15 — Machine totals need phase and topology. Added adjacent ProMCP profiling and a pinned release audit. The total-cost ledger now keeps cold start, discovery, execution phase, effective topology and runtime state visible; no visualization comparison, reproduced run, or cost winner was added.
• 2026-08-15 — Contribution is a state, not a handoff conclusion. Added accepted non-owner AI-assisted work in Prism and OpenClaw, Prism’s repeat contribution, and OpenClaw’s open repair proposal. The evaluator must keep contribution, authority, and recovery separate; no complete lifecycle comparison exists.
• 2026-08-15 — Repaired release separated from independent recovery. Added Prism as a third same-artifact dashboard afterlife and the first held multi-attempt delivery chain. It reaches corrected release and maintainer runtime verification, not independent final acceptance; no complete cost or lifecycle comparison exists.
• 2026-08-15 — Equal rounds are not equal resources. Added METAL as the seventh specialist-cost fragment. Its routes match at five candidates or recurrences but not call topology, and the paper and released logs lack the observed per-arm usage needed for an equal-budget claim. The ledger now requires planned and observed receipts across twelve resource dimensions; no run or cost winner exists.
• 2026-08-15 — Total cost starts before inference. Added a sixth fragment, quarantined its exact figures after four source-integrity failures, and expanded the protocol to eleven lanes, five cost classes, declared amortization, and a source preflight. No route ran and no winner exists.
• 2026-08-15 — One production afterlife, no complete lifecycle. Added KubeStellar’s self-reported same-dashboard regression, two-step repair, restored deploy, and prevention chain plus PM4Py-UCM’s measured refinement effort; preserved the missing whole cost, comparison, handoff, accessible reader, decision, and trust lanes.
• 2026-08-15 — Total workflow cost gets a denominator, not a winner. Audited five primary cost fragments and added a same-task, equal-budget, ten-lane ledger through accepted delivery, reader use, and a later change. The held record supports no summed total or specialist-versus-general ranking.
• 2026-08-15 — Longitudinal learning gap made runnable, not resolved. Added the bounded search null and a three-arm, five-wave creator design with six-month unassisted transfer, a separate blinded reader stage, ten outcome lanes, and ten unfulfilled human/experimental release gates.
• 2026-08-15 — Typed interaction plan added; forecast still unchanged. Added ViviDoc’s State–Render–Transition–Constraint mechanism, same-pipeline ablation, creator/rater evidence, and reader/lifecycle limits; disclosed that it predates the forecast freeze and resolves none of the seven tests.
• 2026-08-15 — Editor calibration task prepared, not performed. Added ten stimulus-by-lens baselines, ten synthetic quality vignettes with two swapped repeats, three response states, 14 rejected adverse records, and seven audience decisions. No editor, appointment, calibration, tie, call, contact, ranking, or result exists.
• 2026-08-15 — Seven R4 receipts defined; all real answers pending. Added three per-slot provider receipts, editor acceptance and calibration, a capped 30-primary-call budget, exact-set release, 20 admissible paths, 16 rejected adverse states, and seven audience decisions. No provider, editor, budget, release, call, contact, ranking, or result exists.
• 2026-08-15 — Two R4 prompts, two schemas, zero model answers. Added separate integrity and readability prompts, three honest raw-response states, twelve rejected aggregate/lens/identity/evidence/state failures, and seven audience decisions. Exact models, editor calibration, budget, release, calls, contacts, rankings, and results remain absent.
• 2026-08-15 — Five R4 inputs frozen; no model run. Added five opaque-ID SVGs, a 19-file model-input allowlist, nine source-evidence files, five evaluator-only truth records, adverse leakage/hash checks, corrected visual-inspection findings, and seven audience decisions. Model slots, editor, tie, prompts, budget, release, calls, and results remain absent.
• 2026-08-15 — R4 is preregistered, not run. Added five candidate contexts, three empty exact-model slots, separate integrity/readability lenses, a human-editor and tie-rule gate, seven audience translations, and a pilot boundary. No stimulus, ranking, call, contact, or divergence exists.
• 2026-08-15 — Outcomes stay separated instead of becoming one score. Added two mechanical tiers, six exact nonmechanical lanes, authority binding, 90 preserved synthetic not-run states, aggregate-score rejection, and seven audience translations. No artifact, arm winner, human result, or efficacy claim is implied.
• 2026-08-15 — Six gates become an attributable decision register. Added 17 admissible paths across four authority roles, fail-closed response and final-release rules, and seven audience translations. All six selections and release remain pending; no provider fact, approval, spend, contact, call, or efficacy result is implied.
• 2026-08-15 — Acceptance authority stays typed and human. Added six nonmechanical authority lanes, five fixture-specific role maps, a 30-state mechanical-only not-run policy, W3C’s conformance/user-evaluation boundary, seven audience decisions, and the research-owner gate. No person, model call, human verdict, or scope selection is implied.
• 2026-08-15 — One-layer candidate separates mechanism from skill bundle. Added one post-artifact evidence-reconciliation candidate, its five-part causal-unit test, five-context failure target, seven audience decisions, and the owner-ratification boundary. No selection, call, or effect is claimed.
• 2026-08-15 — Credential-free runner separates task from evaluator. Added five source checks, 15 isolated fixture-arm envelopes, an explicit model- input/evaluator-only boundary, external artifact/render/trace/check hashing, fail-closed receipt identity, six residual blockers, and seven audience translations. No model, artifact, browser, human, or efficacy result is implied.
• 2026-08-15 — Current-harness manifest preflight. Added the exact fields that can be frozen before a call, an 18-field equivalent-receipt contract, seven named blockers, zero-call/no-spend evidence, and audience-specific rules for interpreting a workflow comparison.
• 2026-08-15 — Interactive-application source packet. Added the fifth of five E0 source packets, including exact canonical state, synchronized surfaces, replay, history, reset, export, keyboard, narrow delivery, stale-selection recovery, and prepared-change gates with explicit browser, maintenance, accessibility, and human-evidence limits.
• 2026-08-15 — Specialized uncertainty source packet. Added the fourth of five E0 source packets, including exact official estimate, margin-of-error, geography, pairwise-test, misleading-rank, refusal, render, and expert-review gates with explicit cross-domain and human-evidence limits.
• 2026-08-15 — Governed-BI source packet. Added the third of five E0 source packets, including exact semantic, vintage, permission, verified-query, refusal, fanout, and incident-recovery gates with explicit production and human-evidence limits.
• 2026-08-15 — Exploratory source packet. Added the second of five E0 source packets, including exact pooled/group checks, revision receipts, audience-specific use boundaries, and an explicit familiar-data limit.
• 2026-08-15 — Shared-baseline readiness gate. Added the reconciled historical custody boundary, the initial context-readiness state, and the equivalent-receipt requirement for bare, normal-harness, and one-layer arms.
• 2026-08-14 — Initial public edition. Established the contextual goals typology, literature synthesis, technique history, benchmark timeline, convergence and divergence findings, demonstrated negatives, and research gap ledger.