We just open-sourced Handit, a tool we built out of frustration with AI agents breaking in production and us not knowing why or how to fix it.
Handit hooks into your logs, evaluates every agent decision, and when something underperforms (drift, hallucination, latency, even tailored key actions), it not only flags it, but suggests and applies a fix (like a better prompt, model, or dataset). Then it A/B tests that fix and keeps the better one live.
We originally built this to debug our own LangChain pipelines, but others started using it too, so we decided to clean it up and make it open source. It works with any LLM system where you have logs (LangGraph, RAGs, etc.), and it's 100% open source—no black boxes.
Would love your feedback,and we're super curious if others have been struggling with the same “drift + broken agent” pain we’ve had.
I've tried this using n8n and it has been amazing! Literally, I'm going out for coffee, and the systems are getting better. Is amazing to see a product like this exists. Congrats on such an unbelievable product, you guys rock!
Hey HN!
We just open-sourced Handit, a tool we built out of frustration with AI agents breaking in production and us not knowing why or how to fix it.
Handit hooks into your logs, evaluates every agent decision, and when something underperforms (drift, hallucination, latency, even tailored key actions), it not only flags it, but suggests and applies a fix (like a better prompt, model, or dataset). Then it A/B tests that fix and keeps the better one live.
We originally built this to debug our own LangChain pipelines, but others started using it too, so we decided to clean it up and make it open source. It works with any LLM system where you have logs (LangGraph, RAGs, etc.), and it's 100% open source—no black boxes.
Would love your feedback,and we're super curious if others have been struggling with the same “drift + broken agent” pain we’ve had.
Repo: https://github.com/Handit-AI/handit.ai Docs to get started: https://docs.handit.ai/ Landing page: https://www.handit.ai/
I've tried this using n8n and it has been amazing! Literally, I'm going out for coffee, and the systems are getting better. Is amazing to see a product like this exists. Congrats on such an unbelievable product, you guys rock!
Thanks! we love seeing people using self improving AI even in no code tools, that's game changer!