Skip to content

Stage 4: Joining and assembly

Stage 4 puts every record under one patient ID, links each document to its patient, and assembles the bundle.

Which ID a record gets

  1. The ID the registry supplies. On the warehouse route this is patient_id in uuid_mapping.csv, looked up by document UUID. On the per-participant route it comes from participant_files.csv or from the folder a file sits in under inputs/fhir/ or inputs/documents/. FHIR exports listed in the mapping file belong to the patient the mapping names, whatever patient ID the health system wrote inside the file.
  2. The DocumentReference index. A document the mapping does not list can take the patient from a FHIR DocumentReference that points to it (see Stage 3).
  3. MRN. A C-CDA document still without a patient is matched on its MRN, compared after removing punctuation and changing to upper case, against patients already known.

A document with no patient after these steps is kept in documents with an empty patient_id, so it can be found and checked without being added to anyone's record.

Demographics

Demographics are filled from FHIR Patient resources first, then from the C-CDA <recordTarget> header, then from any first_name, last_name, mrn, dob or gender columns in uuid_mapping.csv. Each source only fills an empty field; the first non-empty value is kept. The patient ID each health system uses is kept as source_patient_id, and a patient with records from two health systems has both in source_patient_ids.

Merging on name and birth date

Records that arrive without a registry-supplied ID can be merged into a patient whose first name, last name and birth date all match exactly, ignoring case. Two patients whose IDs the registry supplied are never merged with each other; when they match, they are left apart and the log reports it for the team to check.

--no-name-dob-merge turns this merging off. Use it when every file arrives with a registry-supplied ID.

Field aliases

Each record has canonical field names and aliases, so later tools can use whichever they expect:

Canonical Aliases
effective_date effective_datetime; start_date (encounters, medications), authored_on (medications), onset_datetime (problems), recorded_date (allergies, immunizations), performed_datetime (procedures), occurrence_datetime (immunizations)
display_name allergen (allergies), vaccine (immunizations), visit_type_display (encounters)
code allergen_code (allergies), vaccine_code (immunizations)
code_system allergen_system (allergies), vaccine_system (immunizations)

Computed fields

Each patient gets num_documents, the number of documents linked to that patient. The bundle also has labs_vitals, which joins labs and vitals for tools that expect one table.

What to check

The run log and metadata.warnings in dashboard_data.json list identity warnings: name and birth date matches left unmerged, files holding more than one Patient resource, and attachments that were not delivered.