Stage 4: Joining and assembly¶
Stage 4 puts every record under one patient ID, links each document to its patient, and assembles the bundle.
Which ID a record gets¶
- The ID the registry supplies. On the warehouse route this is
patient_idinuuid_mapping.csv, looked up by document UUID. On the per-participant route it comes fromparticipant_files.csvor from the folder a file sits in underinputs/fhir/orinputs/documents/. FHIR exports listed in the mapping file belong to the patient the mapping names, whatever patient ID the health system wrote inside the file. - The DocumentReference index. A document the mapping does not list can take the patient from a FHIR DocumentReference that points to it (see Stage 3).
- MRN. A C-CDA document still without a patient is matched on its MRN, compared after removing punctuation and changing to upper case, against patients already known.
A document with no patient after these steps is kept in documents with an empty patient_id, so it can be found and checked without being added to anyone's record.
Demographics¶
Demographics are filled from FHIR Patient resources first, then from the C-CDA <recordTarget> header, then from any first_name, last_name, mrn, dob or gender columns in uuid_mapping.csv. Each source only fills an empty field; the first non-empty value is kept. The patient ID each health system uses is kept as source_patient_id, and a patient with records from two health systems has both in source_patient_ids.
Merging on name and birth date¶
Records that arrive without a registry-supplied ID can be merged into a patient whose first name, last name and birth date all match exactly, ignoring case. Two patients whose IDs the registry supplied are never merged with each other; when they match, they are left apart and the log reports it for the team to check.
--no-name-dob-merge turns this merging off. Use it when every file arrives with a registry-supplied ID.
Field aliases¶
Each record has canonical field names and aliases, so later tools can use whichever they expect:
| Canonical | Aliases |
|---|---|
effective_date |
effective_datetime; start_date (encounters, medications), authored_on (medications), onset_datetime (problems), recorded_date (allergies, immunizations), performed_datetime (procedures), occurrence_datetime (immunizations) |
display_name |
allergen (allergies), vaccine (immunizations), visit_type_display (encounters) |
code |
allergen_code (allergies), vaccine_code (immunizations) |
code_system |
allergen_system (allergies), vaccine_system (immunizations) |
Computed fields¶
Each patient gets num_documents, the number of documents linked to that patient. The bundle also has labs_vitals, which joins labs and vitals for tools that expect one table.
What to check¶
The run log and metadata.warnings in dashboard_data.json list identity warnings: name and birth date matches left unmerged, files holding more than one Patient resource, and attachments that were not delivered.