There is a version of AI-powered healthcare data interoperability that looks brilliant in a demo. You type a natural language prompt, a configuration materializes, data flows, and the room nods along. It’s clean, it’s fast, and it’s incredibly compelling.
And then it meets production.
This isn’t a critique of the technology—we believe in its potential deeply. My team is currently using Claude to build all kinds of incredible things at a pace I never thought possible (more on that below). The comment is meant to be a reality check. An acknowledgement that, in day-to-day execution, the “last mile” is where healthcare data projects succeed or fail because this is where the vast majority of the complexity lives.
Most AI tools for healthcare data integration are currently optimizing for “time-to-wow”: getting a prototype working fast enough to be impressive. As a buyer, user, or investor, it is easy to be dazzled by the potential. However, the vital question is how the tool and/or the integration performs in production, when it encounters the unfiltered chaos of real-world data.
The myth of the textbook integration
Modern LLMs are genuinely gifted at pattern matching against well-documented standards like HL7v2, C-CDA, and FHIR. Today, anyone with the standard and an idea and a foundational AI model can vibe-code a project in a matter of days that appears functionally correct. It might even have a usable prototype!
The moment this prototype hits a live environment, the vibe-coding begins to fail. Healthcare data rarely survives contact with the standards intended to govern it. EHRs allow for massive customization, and that customization takes forms that no spec documents or foundational model can predict. As the industry likes to say, “if you’ve seen an instance of an EHR at one provider organization, you’ve seen one instance of that EHR.”
Examples of last mile healthcare data integration complexities:
- EHRs routinely append custom Z-segments to otherwise valid HL7v2 messages to carry data that doesn’t fit the standard structure. Those segments are invisible to a parser that doesn’t know they’re there, which means clinically significant data gets silently dropped.
- The FHIR standard explicitly permits customization using FHIR Extensions, and EHRs use them extensively, meaning a FHIR resource can be schema-valid while carrying critical fields that a model trained only on the base standard will simply ignore.
In either case, the message passes validation. The failure remains latent until something downstream breaks. You don’t learn to catch these things from reading the spec. You learn them from years of encountering what the spec looks like after a particular EHR, and a particular provider organization’s configuration of that EHR, implemented it. That’s why there’s no such thing as a “textbook integration.” Even when the standard is used, there’s always a customization. LLM data can help us get to the textbook base configuration incredibly fast. But it takes deep expertise to anticipate exceptions and ensure real-world functionality.
The points above are illustrations, but we are also learning this “last mile” dynamic first hand at Redox. My team is currently building an AI tool that will soon be embedded in the Redox Engine dashboard called the ‘config modifier AI assistant’ (follow us for more info when it is available). The team was able to spin up a working prototype in about a week. One week! It was a real “time-to-wow” moment when the assistant was shown during our engineering sprint demo. The Zoom meeting chat absolutely lit up with excitement and encouragement and calls to SHIP IT!
The vibe coding got us to “wow” incredibly quickly. But before we can roll out a feature to customers, we need to make sure it works in the real-world. The next step was to evaluate the precision and accuracy against production scenarios. As expected, the AI assistant delivered a high success rate for straightforward, textbook configs. However, the more complex configs – the non-textbook ones, similar to those we work on with customers every day – require significant knowledge and tuning to advance the feature to be production ready. We are able to complete this last mile work because we have 12+ years of real-world healthcare data experience, including 15,000+ connections currently live. This is a unique knowledge set that many do not have, and may not know that they need. Preparing beyond the textbook is a true “you don’t know what you don’t know” predicament.
The agent harness is the difference between vibes and real-world performance
Let’s recap: Vibe-coded demos are super impressive and quick to develop. It makes sense why there’s rising industry sentiment that integration development will become much easier in the near future. However, healthcare data integrations are never textbook, and privacy, security, and governance requirements are critical. It begs the questions, “What is the magic required to bridge the gap – using AI tools to move quickly, while also ensuring the final product meets all the real-world performance requirements?” The answer is a combination of context, knowledge, and the agent harness.
As the foundational models e.g. Sonnet 4.6, Opus 4.7, Gemini 3.1 Pro, GPT-5.4; latest versions at the time of publication) continue to evolve faster than the Moore’s Law curve can stay relevant, we predict the performance of these models will converge. The models will become standardized and their differentiation shrinks. The true differentiator for developing autonomous agents will be the agent harness layer (like Claude Code) that sits on top of the models. The agent harness is the layer that is responsible for gathering all the contextual information the model needs, and managing the session and historical state (short term and long term memory) necessary for complex multi-step integration tasks. For healthcare data, the agent harness must incorporate the nuances of various EHR APIs, data transformations (like HL7 v2 to FHIR), and the conditional logic for error handling to generate robust healthcare integrations.
The power of the agent harness layer is evident with our own Redox engineers using Claude Code for day-to-day development. We created close to a hundred Claude.md files across our code repository to provide context on what the code means, and developed more than 10 Claude Skills to teach Claude how to follow our development process. All this context and teaching is the magic that Claude Code needs to reliably tackle the complex tasks that our engineers throw at it each day.
In the same way, building AI powered tooling that generates robust healthcare data integrations requires a lot more than what the foundational model knows. It requires specialized contextual information that goes beyond the standards and textbook integrations. It requires knowledge based on thousands of real-world configurations that have been tested over many years and across 100+ EHRs. Redox is building a “healthcare data integration agent harness” to sit on top of the foundational models, fed by the contextual information we’ve aggregated by experience. Vibe-coding gets us to the demo, our proprietary agent harness is the magic that closes the gap between demo and production.
Four production-ready must-haves
Anonymized real-world data. User permissions. Privacy and quality controls. Audit trails. “Experimental” and “good enough” are NOT good enough when it comes to healthcare data. Given how dazzling many of the “wow” demos are, how can you know if a tool or partner showing off some impressive AI is ready for production?
These are the questions that reveal whether a tool is production-ready:
1. What is the “training” data, and how was it generated? Synthetic data and spec-derived examples will only get you so far.
- Has the model been trained on real-world configurations?
- Was it built against real-world EHR environments?
- Has it been tested with production data?
>>Look for both breadth and depth; this is the single biggest predictor of how the tool will perform in edge cases.
2. How does the tool handle the cases it hasn’t seen before? Any AI tool will perform well on patterns it recognizes. What matters is how it behaves at the boundary; when it encounters something outside its training distribution.
- Does it fail gracefully?
- Does it flag uncertainty?
- Does it have a human escalation path for novel cases?
>>A tool that confidently hallucinates outputs on unfamiliar data is more dangerous than one that acknowledges its limits.
3. What are the privacy, security, and governance controls? An AI tool without HITRUST AI standards, with access to HIPAA data, passively processing data and making changes to models unilaterally – this is the stuff of nightmares.
>>Prompt evals, AI guardrails, and datasets to validate model performance are all non-negotiable. Any AI tool and partner needs to prove the tech meets or exceeds industry requirements.
4. Who is behind the tool, and what experience do they have? This is the least technical question and often the most revealing. AI tools for healthcare interoperability are only as good as the domain expertise embedded in them.
- Who built the training pipeline?
- Who defined what “correct” looks like?
- Who decided which edge cases mattered?
>The team’s accumulated experience is what you’re ultimately buying. Years of direct exposure to how real healthcare data behaves across diverse production environments is not something that can be replicated quickly.
The healthcare organizations that will get the most out of AI-powered interoperability in the next few years won’t necessarily be the ones who moved fastest. They’ll be the ones who asked the right questions early, chose partners with breadth and depth behind their tools, and invested in the unglamorous infrastructure work that makes AI reliable rather than just impressive.
The last mile has always been the hardest part of healthcare interoperability. AI and especially vibe-coding don’t change that. But the right AI, with the right context, built by teams who’ve lived in production, can finally solve the last mile complexities, at scale.
Summary
- The “Time-to-Wow” Trap: While AI can “vibe-code” impressive healthcare data prototypes in days, these often fail in production because they are trained on “textbook” standards (HL7, FHIR) that don’t account for the chaotic, highly customized reality of real-world EHR environments.
- The “Last Mile” Gap: Successful integration requires moving beyond foundational models to an “agent harness” layer. This layer provides the necessary context, historical memory, and deep domain expertise to handle non-standard edge cases and custom data segments that AI alone might ignore or misinterpret.
- Production-Ready Criteria: To move past the “demo” phase, healthcare AI tools must be evaluated on four pillars: the quality of their real-world training data, their ability to fail gracefully in unfamiliar scenarios, rigorous security/governance controls (like HITRUST), and the depth of the expertise behind the model.
- The Redox Approach: Redox is leveraging its 12 years of experience and 15,000+ live connections to build a specialized agent harness. This ensures that their AI tools are fed by real-world configurations rather than just synthetic data or spec documents.
This post was written by Sasi Mukkamala, Redox’s Chief Technology Officer.