Redox and Databricks: Powering advanced analytics with healthcare data
Aug 08, 2023
Healthcare is sitting on mountains of data
Pop quiz: Which industry accounts for about 30% of newly created data around the world and is generating more data faster than any other industry? You guessed it — healthcare.
But as healthcare organizations know, having a lot of data does not mean that it is useful for decision making. The truth is most of this data remains largely untapped. Data is often unstructured (e.g., image scans, clinician notes), comes in different formats (e.g., JSON, XML, pipe delimited) and is siloed within a myriad of source systems like electronic health records (EHR), medical imaging databases, claims processing systems, revenue cycle solutions, health information exchanges (HIE), and so on. There is tremendous promise around technology’s ability to transform healthcare, we must solve healthcare’s mulitmodel data problem to fully unlock the potential. So how do healthcare organizations bring together data from different sources, in different formats, to form a holistic understanding of the patient and subsequently use predictive AI to make informed decisions and generate better outcomes?
This is why we are so excited to announce a new partnership with Databricks. Redox + Databricks Lakehouse can upend the status quo by making it simple, fast, and efficient to take multiformat, structured, and unstructured healthcare data into a single platform for all downstream data use cases (from BI reporting through complex machine learning) while maintaining fully compliant data governance and visibility.
In this blog, we will address the challenges that our customers face, how Redox and Databricks can help, and where you can get started.
Data remains largely untapped
For healthcare providers and payers, the ability to unify disparate data sources and to democratize that data across the business can improve outcomes, reduce costs, and increase agility. It’s playing out today in ways we couldn’t have imagined years ago.
Not only do we need to overcome data siloed within a myriad of source systems, we must find ways to then bring together high volumes of highly variable data in a secure, compliant way that respects patient confidentiality and meets regulatory requirements. These data hurdles have hindered progress for healthcare organizations to see the patient and overall environment holistically.
With Redox, healthcare organizations can now unify all of their data within Databricks, make it available for internal and external stakeholder collaboration for both real time and batch, and create a closed feedback loop back into an EHR or other operational systems.
In-house integrations remain lengthy and costly
We often hear from our customers that the lack of interoperability and data sharing between various systems, applications, and devices remains the most significant obstacle in extracting value from all of their data, sharing information with key stakeholders (both internal and external) and gaining a holistic view of the patient and operations.
A lack of standardization across data formats, protocols, and security measures leads to the need for a specialized skill set to understand all of the underlying systems.
Despite standards like HL7® Fast Healthcare Interoperability Resources (FHIR®) and 21st Century Cures Act regulations emerging to help increase interoperability, many standards and file formats still exist across source systems spanning electronic health records (EHR), regional health information exchanges (HIE), digital health products, and beyond.
To tap into this data, organizations need to design for a variety of authentication methods - the many variants of OAuth, custom EHR authentication schemes spanning TLS client certificate, basic authentication, JWT, and SMART.
It requires understanding and communicating with source data via SFTP, SOAP, OIDC, SAML, Webhooks, and asynchronous APIs. Lastly, a plethora of data standards exist - FHIR, HL7v2, X12, CDA, DICOM, NCPDP - across varying formats like XML, JSON, pipe delimited, and CSV flat files.
All of this is costly and time-consuming to build in-house. In addition to building their data pipelines, engineering resources have to continuously maintain a variety of integrations. For many of the healthcare provider and payer organizations we work with, they do not have available resourcing, nor the time to accomplish such a monumental lift.
Redox changes that.
Unifying healthcare data
Redox is an interoperability platform that helps payers, providers, and products connect to EHRs and every other healthcare data source. Redox integrates with over 5,500 healthcare organizations (including the top imaging and diagnostic centers), connects with 90+ EHR vendors, and has processed over 42 billion messages.
How does it work? Redox acts as a neutral intermediary, standardizing and translating data between disparate healthcare systems. By providing a single point of bi-directional integration, Redox simplifies data discovery and extraction, and ensures compatibility between different applications, electronic health records (EHRs), and healthcare provider and payer organizations.
With advanced product functionality, experience in healthcare, and in-depth interoperability expertise spanning legacy standards to FHIR, customers get from project kick-off to production in weeks, not months.
Databricks: Unlocking data insights and use cases
Once the data is sourced, normalized, and made available via Redox, the Databricks Lakehouse Platform ingests this data – as a batch or continuous streaming — for downstream analytics. Databricks supports both reading and writing of all FHIR resources in standard FHIR Bundle exchanges. That enables quick onboarding of data, opening the door for real-time BI and AI use cases in the Lakehouse, such as optimizing where resources and staff are directed based on real-time hospital capacity.
The Databricks Lakehouse unifies data, analytics, and AI under a centralized platform, simplifying and accelerating data processing while also facilitating advanced analytics and machine learning.
Simplified governance and compliance with Databricks
What makes Databricks Lakehouse particularly well suited for the healthcare industry is the centralized access control, auditing, lineage, and data discovery capabilities provided by the Unity Catalog. In this way, information sharing and collaboration are enabled for faster innovation while still maintaining strict access control down to row and column granularity to meet regulatory requirements.
What’s next for your organization?
Interoperability is the key to unlocking the true potential of healthcare data. Redox provides a standardized approach to integrate disparate systems and foster seamless data exchange. The Databricks Lakehouse in turn unifies this data and makes this data immediately available for use in BI, analytics, and advanced ML. By combining Redox's integration capabilities tailored to healthcare data sources with Databricks' data processing and analytics prowess, healthcare organizations gain the ability to deliver better decision-making, more visibility into operations, and improved patient outcomes with personalized care.
Request a demo today.