Developers can now debug and evaluate AI agents locally with Raindrop's open source tool Workshop

Developers can now debug and evaluate AI agents locally with Raindrop's open source tool Workshop

Observability startup Raindrop AI’s new open source, MIT Licensed "Workshop" tool, launched today, gives developers something that they've likely wanted, perhaps subconsciously, since the agentic AI era kicked off in earnest last year: a local debugger and evaluation tool specifically designed for AI agents, allowing devs to see all the traces of what their agent has been doing in a single, lightweight Structured Query Language (SQL) database file (.db)It functions as a local daemon and UI that streams every token, tool call, and decision to a local dashboard—typically hosted at localhost:5899—the moment it occurs. By visiting their localhost, developers can then see everything their agent was up to — including mistakes or errors — and identify what went wrong, when, and ideally, discern why. It's all stored in a single .db file, which takes up relatively little memory, according to a X direct message VentureBeat received from Ben Hylak, Raindrop's co-founder and CTO (and a former Apple and SpaceX engineer). This real-time telemetry eliminates the latency of traditional polling and addresses a growing developer concern regarding the privacy of sending local traces to external servers.The tool is available for macOS, Linux, and Windows. It can be installed through a one-line shell command that automates binary placement and PATH configuration for bash, zsh, and fish shells. For developers who prefer to build from source, the repository is hosted on GitHub and utilizes the Bun runtime. The product: establishing a self-healing eval loopThe platform’s standout feature is the "self-healing eval loop," which allows coding agents like Claude Code to read traces, write evals against the codebase, and fix broken code autonomously. In a practical application, if a veterinary assistant agent fails to ask necessary follow-up questions, Workshop captures the full trajectory. Claude Code then reads this trace, writes a specific eval, identifies the logic error in the prompt or code, and re-runs the agent until all assertions pass.Compatibility and ecosystem integrationWorkshop is compatible with a broad range of programming languages, including TypeScript, Python, Rust, and Go.It integrates with popular SDKs and frameworks such as the Vercel AI SDK, OpenAI, Anthropic, LangChain, LlamaIndex, and CrewAI. It is also designed to work seamlessly with various coding agents, including Claude Code, Cursor, Devin, and OpenCode.Licensing and community implicationsWorkshop is released under the MIT License, ensuring it remains free and open-source for all users. This permissive licensing is intended to foster community contribution and allow enterprise users to maintain data sovereignty. Hylak noted on X that the tool was built to provide a "sane" way to debug agents locally, changing how their team and early customers build autonomous systems.To celebrate the launch, Raindrop offered limited-edition physical merchandise to users who installed the tool and executed a specific "drip" command.]]>

Source

Original Article: Developers can now debug and evaluate AI agents locally with Raindrop's open source tool Workshop

Published: May 15, 2026

Author: carl.franzen@venturebeat.com (Carl Franzen)

---

Disclaimer: This article is for informational purposes only and does not constitute financial advice. Consult a licensed financial advisor before making investment decisions.