Quickstart¶
Run the pipeline end to end in a few minutes with no real EHR access: first on the single-patient sample data, then on the invented ALS cohort with every add-on module. Where to run it covers doing the same in Jupyter, Google Colab or AWS.
1. Get the code¶
Download RegistryForge.zip and unzip it:
RegistryForge/
├── pipeline/ run_pipeline.py, run_all.py, the add-on modules, dashboard.html
├── samples/ the single-patient sample export (ccda_chunks.csv, fhir_chunks.csv, uuid_mapping.csv)
├── examples/ make_synthetic_cohort.py, the invented ALS cohort generator
├── notebooks/ run_registry_forge.ipynb
├── templates/ vital_status.csv
└── tests/
2. Install the packages¶
cd RegistryForge
python -m pip install -r requirements.txt
python -m pytest -q tests # optional: confirms the install, on invented records only
3. Run the core pipeline on the sample data¶
The pipeline reads a working directory laid out the way the Databricks export writes it:
registry-forge-demo/
├── CCDA and FHIR data/
│ ├── ccda_chunks.csv
│ └── fhir_chunks.csv
└── uuid_mapping.csv
mkdir -p "registry-forge-demo/CCDA and FHIR data"
cp samples/ccda_chunks.csv samples/fhir_chunks.csv "registry-forge-demo/CCDA and FHIR data/"
cp samples/uuid_mapping.csv registry-forge-demo/
python pipeline/run_pipeline.py --base-dir registry-forge-demo
Expected output (shortened):
[12:00:00] Registry Forge: starting
[12:00:00] Mapping CSV: uuid_mapping.csv (1 rows)
STAGE 1: Decoding & reassembly (warehouse chunk exports, if present)
[12:00:00] Wrote 1 files to registry-forge-demo/ccda_assembled/
[12:00:00] Wrote 1 files to registry-forge-demo/fhir_assembled/
STAGE 2+3: Parsing FHIR and documents
[12:00:00] Format counts: {'fhir_attachment:text': 3, 'ccda_xml': 1}
[12:00:00] problems: 6 records
[12:00:00] medications: 7 records
[12:00:00] ...
STAGE 4: Assembling bundle
STAGE 5: Enriching display names
STAGE 6: Filtering test patients
STAGE 7: Building code inventory
The working directory now holds dashboard_data.json, patient_master.csv, code_inventory.csv, note_extractions.csv and the run log.
4. Run everything on the invented cohort¶
The generator writes invented ALS participants (Jane and John Doe shells) in the same export layout, with C-CDA, RTF, PDF, HTML and text documents, FHIR bundles from one or two invented health systems, and a vital_status.csv. run_all.py runs the core pipeline and every add-on module:
python examples/make_synthetic_cohort.py --out practice_run
python pipeline/run_all.py --base-dir practice_run --cutoff 2026-10-31 --vital-status practice_run/vital_status.csv
--cutoff is the data cut-off: follow-up for survival summaries ends there, so the same data always give the same figures. Outputs are sorted into three places by privacy level:
| Folder | What it holds |
|---|---|
practice_run/ |
Identified core outputs: dashboard_data.json, patient_master.csv, note_extractions.csv and the rest |
practice_run/restricted/ |
Row-level files with patient IDs or text excerpts (device, exposure and drug repurposing tables, Phenopackets, OMOP tables): they stay inside the approved environment |
practice_run/reports/ |
Counts-only reports with counts below 5 held back: reviewed in full before anything is shared |
Add --vocab /path/to/athena to write OMOP tables, and --with-excerpts to write review dashboards with masked excerpts into restricted/.
5. Look at the results¶
Open pipeline/dashboard.html in a browser, click Load dashboard_data.json and choose the file from the working directory. Identifiers are hidden by default. Open the HTML files in reports/ for the cohort summary, note extraction, device, exposure and drug repurposing reports.
6. Move to your own data¶
Replace the sample files with the output of the Databricks export in a working directory inside an approved environment, add vital_status.csv if deaths come from outside the records, and run run_all.py the same way. Check Where to run it first, and add rules for test records to test_patients.txt (see Stage 6).