Successfully migrating from an on-prem setup to a cloud data lake
Context
Our US-based client is a leading provider of emergency department and other healthcare services.
They outsource physicians in emergency medicine, hospital medicine, anesthesiology, critical care, obstetrics, orthopedic surgery, general surgery, ambulatory care, post-acute care, and medical call centers to approximately 2,900 acute and post-acute facilities nationwide.
Business problem
The client's existing data analytics stack was scattered across several disparate data sources leading to inefficient data visualization and deriving necessary, actionable insights from those sources. Further, the overall scalability of the platform was suboptimal.
Expected goals
Key goals expected from this engagement included:
To implement a scalable, cloud-based solution offering configuration-based data ingestion
Shorter, optimized implementation cycles for integrating new data sources
Solution
Data integration across disparate sources
The solution was delivered to the client included:
Data migration with multiple ETL solutions into a multi-cloud infrastructure
Improved data security through integration across 28 disparate data sources
Deployment of Snowflake for enhanced data visualization capabilities and improved, actionable reporting
The solution was deployed using:
ETL Process: RDMS, CDC, Pipeline, Python, Cloud-based services
Cloud hosting: AWS Infrastructure + MS Azure DevOps
Security and integration: OKTA
Data analytics: Snowflake
Outcomes of the Engagement
Efficiency gains due to cloud migration from on-prem hosting
Improved data visualization across disparate sources
The solution helped the client to extract data from 28 disparate sources through ingestion either through the cloud solution or raw import formats
Migration of on-premises analytics stack to the cloud
Optimized the efficiency of internal business processes by decommissioning on-premises requirements to a more secure cloud-based infrastructure
Save this case study for future reference
Key project outcome metrics:
Sources for data collection unified