Building a Scalable Data Ingestion Architecture | Komodo Health
Frost & Sullivan Names Komodo Health RWE Company of the Year

Building a Scalable Data Ingestion Architecture

By Usha Periyanayagam, MD, MPH, and Lucy Chen

At Komodo Health, our Data Ingestion team is responsible for quickly incorporating new data sources into our expanding Healthcare Map™.  One of our engineers, Johannes Leppä, PhD, gave a talk at the DataCouncil conference in San Francisco in April about how we build our scalable data ingestion architecture and some of the lessons learned along the way.   Some of the technologies we use are Spark, Airflow and Kuberenetes.

Here is the video of that talk

Enjoy!  If you are interested in working on this and many other interesting technical problems while helping reduce the global burden of disease we are hiring for a number of roles.

New at Komodo Health

Meet Jason Prestinario. He describes what it’s like leading Komodo Health’s newest division, Healthcare Solutions, and shares his passion for […]
Read More →

Researchers from the MIT Sloan School of Management recently published a breakthrough study on the success rate of clinical trials. […]
Read More →

By most accounts, telemedicine was one of the bright spots in the COVID-19 crisis. The technology helped ease the burden […]
Read More →

Ready to find out more?
We'd love to hear about what challenges you face and how access to healthcare's ground truth can help.