Jun 26, 2026
|
47 min
|
179 views
Agents for Correct, Transparent, and Reproducible Data Analysis - Simon Couch & Sara Altman
Agents for Correct, Transparent, and Reproducible Data Analysis - Simon Couch & Sara Altman (Posit)
Abstract: How do we build competent data analysis agents? Data analysis requires a willingness to pause, question conclusions, and dig into subtleties. Frontier LLMs, however, are optimized to push tasks toward completion, not to slow down when something seems off. This tendency works well for coding agents, where success is often verifiable. But for data analysis, verification is more complicated, and autonomous work by the agent can be at odds with the spirit of the discipline. Drawing on our experience building data analysis agents, we’ll share evaluations that expose where LLM-driven analysis goes wrong and design patterns that keep analyses correct, transparent, and reproducible.
Resources mentioned in the session:
Presentation Slides: https://simonpcouch.github.io/gen-ai-pharma-26 Presentation GitHub repository: https://github.com/simonpcouch/gen-ai-pharma-26 bluffbench: https://github.com/simonpcouch/bluffbench Posit Assistant Terminal (TUI): https://posit-dev.github.io/assistant/docs/downloads/tui/ Posit AI Newsletter: https://opensource.posit.co/tags/ai-newsletter/ Speakers:
Simon Couch builds tools that make the work of data science more joyful and effective. As an engineer on the AI Core Team at Posit, his work spans coding agents, model evaluations, inference engineering, and next-edit-suggestion systems. Drawing on his background in statistics, Simon spent several years authoring and maintaining core packages in the open-source tidymodels framework—like stacks, broom, and infer — before shifting his focus to LLMs. He blogs about his work at simonpcouch.com. Simon authors the Posit AI Newsletter along with Sara Altman.
Sara Altman is a Senior Developer Advocate on the AI Core team at Posit, where she focuses on how AI can be effectively and responsibly used for data science. Previously, she helped build Posit Academy and taught data science and R at Stanford. Sara authors the Posit AI Newsletter along with Simon Couch.
Presented at the 2026 R/Pharma GenAI Day
infer
tidymodels
