Measured evidence
The studies below answer different questions. Their denominators and limits remain attached; the numbers should not be pooled into one score.
The exercise compared integrated ChatGPT analysis with an R/Stata-plus-ChatGPT workflow on simulated epidemiological data. Overall scores were not significantly different.
Twenty visualization novices completed 60 ChatGPT sessions producing 175 charts. The study observed prompting, chart quality, interpretation, verification, and repair—not just whether an image appeared.
Fatal task noncompliance52 of 60 sessions
Incorrect insight recorded12 of 60 sessions
Unusable chart22 of 175 charts
Meanwhile mean confidence was 3.73/5 and satisfaction 3.93/5. Only three verification attempts were observed.
Newer models produced fewer design flaws on replayed initial prompts, but richer interfaces added latency, broken controls, blank renders, and unverifiable interpretations. Better generation changed the failure surface; it did not remove the verification problem.
Use of explanation, code, original and intermediate data, results, and visual summaries
People began with procedure, then often moved to data after noticing trouble; expertise shaped the artifact they trusted
Prepared tasks at one company; data transformations rather than complete visualization projects
Mixed direct manipulation and natural language on reproduction tasks
All completed; visible transformed data, code, explanations, history, and branches supported different verification styles
Six needed hints; no baseline, open exploration, self-owned data, or long-term use
Rapid mockup generation, structured edits, and comparison with a lightly taught Tableau condition
Participants valued speed, simulated data, history, and a concrete object for negotiation
Pre-data prototypes, not analytical correctness, deployment, or ongoing dashboard use
AI versus human-expert design advice, ratings, interviews, and practitioner preference
AI helped enumerate ideas; practitioners preferred experts for accuracy, context, adaptability, and actionable advice
Practitioner sessions used 2023-era GPT-3.5; raw capability comparison is dated
At least five versions of a self-selected visualization over a three-to-five-day window with LLM and perceptual critique
Final work improved 3.69/5 on average; intermediate and expert designers converted feedback into useful edits more readily
Roughly 90–150 minutes of observed work each; no critique baseline, production delivery, or later maintenance
Correction, premature acceptance, non-completion, task time, hints, and perceived control across conversational and decomposed interfaces
Seven episodes were not completed and 31 were declared complete with an issue remaining; structure improved perceived control, not detected success or time
Tasks were engineered to contain model errors and stopped at 15 minutes; no publication, maintenance, or field abandonment
Analyst verification ↗ · Data Formulator 2 ↗ · DashChat ↗ · Visualization advice ↗ · Visualizationary ↗ · Steering and verification ↗