Biopharma is eager to reap all the advantages that AI offers, and several top 20 companies are actively exploring or piloting the use of foundational LLMs for select processes. Will the complexity of healthcare data thwart success?
General large language models (LLMs) have demonstrated remarkable capabilities across many domains, so retrofitting them for healthcare analytics might appear to be an efficient approach to incorporating AI into your organization. However, the many complexities and nuances of working with healthcare data make this a formidable challenge. Success hinges on one critical factor: context engineering expertise — knowing the right context to engineer for producing the best result.
No Substitutes for Healthcare Data Expertise
Context engineering is extremely complex, and its success depends on the knowledge and real-world experience of those building the instruction set. The greater their expertise, the more granular and complete the analysis will be.
Ensuring that a foundational LLM has access to requisite, high-quality data sources is not enough. While high-quality data is essential, it is only the foundation. The AI must be trained to reason like a seasoned healthcare analyst, with the ability to “think” critically — synthesizing myriad data and determining what is relevant/irrelevant for each analysis, recommending the best analytical approaches/methods, surfacing salient insights, refining its own work, and recommending the most opportune interventions.
Nor is increased processing capacity a remedy for the inherent complexity of healthcare data. In fact, recent research shows that, though foundational models have increased their context window sizes dramatically (allowing them to process more text at once), this “cramming” of more information into these windows actually degrades performance. Models suffer from “attention capacity” limitations: Their ability to maintain focus deteriorates, which leads to two critical problems:
- Hallucination where data is missing or ambiguous: Generic AI tends to “fill in” with plausible but potentially incorrect information, creating a dangerous illusion of completeness.
- Quality deterioration with information overload: As more content is pushed into the context window, the model’s attention becomes diffused, compromising its ability to identify and prioritize the most relevant clinical information.
Comparing Foundational vs. Purpose-Built AI Insights
To illustrate, I fed this prompt to both a foundational LLM equipped with the basic tools of web search, data query, and charting capabilities and to Marmot™, Komodo’s AI platform that’s purpose-built for the Life Sciences and healthcare industries. (Note that both had access to the same data sources):
“I’d like to understand the pre- and post-diagnosis patient journey for multiple myeloma (MM) patients. Include analyses that you believe would be helpful in understanding the MM patient journey.”
The foundational LLM generated this Sankey diagram that illustrates the complex journey of MM patients from pre-diagnosis through various treatment phases to advanced disease stages. The width of each flow represents the relative proportion of patients following each pathway.
While the foundational LLM accurately depicted the MM patient journey, its output was mechanical and void of interpretation, as evidenced by the pre-diagnosis and general treatment pathways insights.
Pre-Diagnosis Journey |
First-Line Treatment Considerations |
Symptoms and risk factors
Diagnostic criteria (from clinical trial eligibility) and key diagnostic tests
|
Transplant-eligible pathway
Induction therapy:
Maintenance therapy:
Transplant-ineligible pathway
|
View the full analysis generated by the foundational LLM
In contrast, Marmot shared a much more complete and sophisticated analysis of the patient journey. It included numerous visualizations that conveyed pre-diagnosis symptom patterns, utilization/frequency/timing of diagnostic procedures, HCP specialty involvement, post-diagnosis treatment patterns, and demographic disparities across the patient journey.
Importantly, Marmot gleaned the most salient insights, highlighting significant opportunities for earlier diagnosis and more equitable treatment across demographic segments, particularly for older patients and those with Medicare insurance — and suggests early intervention opportunities based on symptom patterns:
Pre-Diagnosis Journey |
Recommendations |
Symptoms and timeline
Diagnostic procedures
DiagnosisProvider specialties
|
|
View the full analysis generated by Marmot
The Best Path Forward
While foundational LLMs excel at natural language understanding and generation, they lack the rigorous analytical methodologies and specialized training that is required to generate actionable healthcare analytics. At Komodo, we’ve leveraged decades of industry expertise acquired by our product, engineering, research, and analytics teams to address these shortcomings. Our highly curated data, rigorous analytics methods, and knowledge gained from the 1+ million cohorts built with our software are the foundation for Marmot, the healthcare industry’s first AI thought partner.
Learn more by watching our on-demand webinar, where we introduce Marmot and share a live demo.
To see more articles like this, follow Komodo Health on LinkedIn, YouTube, or X, and visit our Resources Hub.
