Configuration

Paths and run settings are command-line options; there is no file to edit before a run. The constants near the top of run_pipeline.py hold defaults and lookup tables you can extend.

Command-line options

run_all.py

Option	What it does
--base-dir DIR	The working directory (required). It holds CCDA and FHIR data/ and uuid_mapping.csv, or files already split by patient under inputs/
--cutoff YYYY-MM-DD	The data cut-off; needed for the drug repurposing module, and used as the reference date for ages in the cohort summary
--vital-status FILE	vital_status.csv with death and last confirmed alive dates from outside the records (default: vital_status.csv in the working directory, if present)
--vocab DIR	An Athena vocabulary download; adds the OMOP tables
--k N	Small-cell threshold for every report (default 5)
--cohort-name TEXT	Name shown on the reports
--with-excerpts	Also write review dashboards with masked excerpts into restricted/
--no-name-dob-merge	Never merge records on name and birth date
--skip MODULE ...	Leave out devices, exposures, repurposing, phenopackets or omop

run_pipeline.py

Option	What it does
--base-dir DIR	The working directory (default: the current folder)
--drive-dir DIR	Optional second folder for uuid_mapping.csv and test_patients.txt, for example a mounted cloud folder
--no-name-dob-merge	Never merge records on name and birth date
--skip-note-extraction	Skip Stage 7c

Every add-on module also runs on its own; python <module>.py --help lists its options.

Paths inside the working directory

Every path is derived from --base-dir:

Path	Read or written
CCDA and FHIR data/ccda_chunks.csv, CCDA and FHIR data/fhir_chunks.csv	read: the chunked export
uuid_mapping.csv (or document_patient_mapping.csv)	read: document-to-patient mapping
inputs/fhir/<patient_id>/, inputs/documents/<patient_id>/	read: files already split by patient, as an alternative to the export
test_patients.txt	read: exclusion rules
ccda_assembled/, fhir_assembled/	written: reassembled documents and bundles
dashboard_data.json, dashboard_data.xlsx, dashboard_data.prefilter.json, csv_exports/, pipeline_run.log	written: core outputs
patient_master.csv, code_inventory.csv, note_extractions.csv	written: Stage 7 outputs

Mapping file

uuid_mapping.csv must have document_uuid and patient_id columns, in any order. Optional columns: first_name, last_name, mrn, dob, gender. The document column may also be named id, doc_uuid or fhirdocument_id. The patient ID in the mapping decides who a document belongs to, including FHIR exports listed in it, whatever patient ID the health system wrote inside the file.

Test-patient exclusions

Two sources, both always applied:

Hardcoded list

Near the top of run_pipeline.py:

HARDCODED_TEST_EXCLUSIONS = [
    # 'name: Test Patient',
    # 'name_contains: zztest',
]

File-based

The pipeline reads test_patients.txt from the working directory (or --drive-dir). Its rules are merged with the hardcoded list. See Stage 6 for the rule syntax.

Code-system preferences

Which coding to pick when a CodeableConcept has several parallel codings:

CODE_SYSTEM_PREF = {
    'medication':   ['rxnorm', '2.16.840.1.113883.6.88'],
    'problem':      ['snomed', '2.16.840.1.113883.6.96',
                     'icd-10', '2.16.840.1.113883.6.90'],
    'lab':          ['loinc',  '2.16.840.1.113883.6.1'],
    # ...
}

To prefer ICD-10 over SNOMED for problems, swap the order. The pipeline matches by substring against both the system name (for example "SNOMED-CT") and the OID, so either works. Every coding is kept in all_codings whichever is preferred.

Display-enrichment tables

LOINC_DISPLAY and SNOMED_DISPLAY are plain Python dicts. Add or remove entries directly:

LOINC_DISPLAY = {
    '8480-6':  'Systolic blood pressure',
    '82953-1': 'ALSFRS-R total score',
    # add your codes here
}

Section template OIDs

C-CDA section template IDs:

TPL = {
    'problems_section':   '2.16.840.1.113883.10.20.22.2.5',
    'medications_section':'2.16.840.1.113883.10.20.22.2.1',
    # ...
}

These are HL7 CDA R2 standard OIDs and rarely need changing. If your warehouse uses non-standard template IDs, override them here.

Name and birth date merging

DEDUPE_BY_NAME_DOB = True lets a record that arrives without a mapped patient ID merge into a patient with exactly the same name and birth date. Two patients the mapping identifies are never merged; the log reports the match for review. --no-name-dob-merge turns merging off.