A large tuberculosis active-case-finding campaign in Cameroon screens community members, primary-health facilities and prisons across seven regions, capturing data in five separate REDCap databases plus several external laboratory and programme systems.
Data for the same participant was fragmented across five REDCap projects with dual and quadruple QR-code identifiers, bilingual French/English forms, and external GeneXpert, treatment-outcome and national-programme feeds — making reliable, timely reporting almost impossible.
I built a three-stage Python pipeline (import → process → report) with an ID-validation engine that regex-checks and repairs every QR code, links screening to specimen to laboratory result, normalises bilingual fields, extracts spirometry metrics from PDFs, and integrates four external data sources — orchestrated hourly through Airflow.
The programme now runs on a single unified participant dataset refreshed automatically through the day, turning five disconnected databases and dozens of raw exports into analysis-ready reporting on screening, eligibility and diagnostic results.