Here's the pattern playing out across the industry right now: someone downloads an open-source agentic AI framework, asks an AI coding assistant to build something quickly, and within hours has a demo that looks like a finished product. It's a real phenomenon with a name: vibe coding. And it's convincing enough that plenty of smart teams walk away thinking they're a few weeks from production. They aren't.
What vibe coding gets wrong about confidence versus readiness #
Vibe coding is what happens when someone spins up a coding assistant, asks it to build agentic AI functionality, and watches it work. The result often looks like a Ferrari. Confirmation bias does the rest: once something demos well, it's tempting to believe it's ready to ship. But a demo answers one question only: whether the concept works in a controlled environment. It says nothing about whether the system understands your business, complies with your regulators or survives contact with real production data.
Why AutoGen, CrewAI and LangGraph fall short in production #
Open-source frameworks are genuinely useful for prototyping, and that's exactly where their value ends for a regulated industry. A few gaps show up consistently once insurers try to take one live:
- Poor integration with legacy core systems, since most frameworks were never built with policy administration or claims platforms in mind.
- Overfitting to one use case, so the framework understands the specific workflow it was built for and nothing about the broader business.
- Missing explainability and traceability, which conflicts directly with what AI-enabled insurance platforms are expected to provide regulators.
- No embedded human feedback loop, so there's no structured way for underwriters or claims staff to correct or coach the system over time.
- IP and licensing exposure, since AI-generated code can pull in unknown open-source libraries without anyone checking what's actually in the stack.
- No out-of-the-box industry fundamentals, such as state-specific underwriting rules, so someone still has to build all of that from scratch.
The real work is the 95%: drift, governance and support #
Even once a framework clears those gaps, the harder problem is what happens after launch. Large language models change behaviour between versions, sometimes without much warning, and a system that performed well on day one can quietly drift from that baseline. Insurers need a way to measure that drift, monitor consistency, and route problems to a real support team the same way any other critical production software would be supported. Skipping this step is why so many promising pilots stall before they ever reach scale.
Recent research puts a number on this: generative AI (GenAI) investment surged sharply in 2025, yet the majority of scaling barriers insurers report have nothing to do with the technology itself. Culture, process and data readiness are the actual bottleneck, which is precisely the 95% a framework alone can't solve.
How to move from proof of concept to production the right way #
Getting an artificial intelligence (AI) framework demo to work is the easy part. Getting insurance-specific decisioning into safe, governed production is a different exercise entirely, and it usually requires:
- Shadowing humans first, running the AI in parallel with real staff decisions before it's allowed to act on its own.
- Measuring drift against a fixed baseline every time the model answers, not just at launch.
- Building in override and escalation paths, so a person can always intervene and the reasoning stays traceable.
- Choosing a vendor built for the industry rather than a generic AI platform, so state-specific rules and compliance logic don't have to be rebuilt from scratch.
- Treating employee adoption as part of production readiness, not an afterthought, since staff who feel replaced won't give the AI the feedback it needs to improve.
Conclusion #
The frameworks aren't the problem. The assumption that a working demo equals a production-ready system is. Insurers that treat the last 95%, governance, drift monitoring, human review, and support, as core project scope get to production. Insurers that treat it as a footnote end up back at proof of concept.
For a closer look at what separates a real build from a demo, and where building versus buying actually makes sense, watch the full conversation between Fabio Sarrico of Celent and Ghassan Karam of Equisoft.
