Skip to content

Troubleshooting

No patients · Bracketed names · Medication codes · Documents without a patient · Empty PDFs · Excel export · Unwanted merges · Test patients · Checks in Python

No patients in the output

Check, in order:

  1. Did stage 1 write files to <base-dir>/ccda_assembled/ and <base-dir>/fhir_assembled/? If not, the chunk files are missing, empty or not in <base-dir>/CCDA and FHIR data/. On the per-participant route, check inputs/fhir/<patient_id>/ and inputs/documents/<patient_id>/.
  2. Did stage 3 find patients? The log line Patients discovered (pre-mapping fill): N gives the count.
  3. Are the document UUIDs in uuid_mapping.csv? A document with no mapped patient and no MRN match is kept without a patient.
  4. If there are no FHIR Patient resources, patients come from the C-CDA <recordTarget> header and the mapping file only; check that the header has recordTarget/patientRole/patient/name.

Names appear in brackets

Names such as ['Smith'] or b'Smith' mean the warehouse export wrote a Python list or bytes representation instead of the text. clean_name removes these, but the export should write plain strings in first_name and last_name.

Medications show no RxNorm code

Stage 3 prefers RxNorm, but it can only choose a code the record holds. Look at all_codings on a sample medication record; if RxNorm is not there, the source sends NDC or local codes only, and the export or the source system needs to include RxNorm.

Documents without a patient

A document has an empty patient_id when none of the linkage steps in Stage 4 found its patient. In order of likelihood:

  1. The document UUID is not in uuid_mapping.csv. Compare the number of mapping rows with the number of documents exported.
  2. The document is a C-CDA whose <recordTarget> is missing or malformed.
  3. The MRN fallback found no match. Compare a C-CDA MRN with a known patient's MRN; both are compared after removing punctuation and changing to upper case.

Every PDF shows no extractable text

The PDFs are most likely scanned images with no text layer and need optical character recognition (OCR). Open one and try to select text; if nothing selects, there is no text to extract.

If a PDF has selectable text in a viewer but none in the bundle, check that its bytes arrived intact. A chunk file re-encoded as UTF-8 anywhere along the way (in the warehouse, in transfer or in storage) damages binary PDF content.

The Excel export fails on a large bundle

dashboard_data.xlsx is written with openpyxl, which is slow and uses a lot of memory for very large workbooks. Use the CSV files in csv_exports/ instead, run a smaller cohort, or change the writer in write_outputs to a streaming one such as xlsxwriter with constant_memory=True.

Two people were merged

Records that arrive without a registry-supplied patient ID can be merged on exact first name, last name and birth date. Two different people with the same name and birth date would be merged wrongly. Run with --no-name-dob-merge to turn merging off, and make sure every document is listed in uuid_mapping.csv. Two patients whose IDs the registry supplied are never merged with each other.

Stage 6 removed no test patients

Check the log line Total active exclusion rules: N. If N is 0, test_patients.txt was not found in the working directory, every line in it is a comment, and HARDCODED_TEST_EXCLUSIONS is empty.

If N is more than 0 but the log says Removed 0 test patients, no rule matched. name: needs an exact full name, name_contains: matches part of a first or last name, both ignore case, and mrn_contains: checks the MRN, not the patient ID. See Stage 6.

Checks in Python

Run these inside the approved environment; the bundle is identified.

import json
b = json.load(open('dashboard_data.json', encoding='utf-8'))

# PDFs with text
pdfs = [d for d in b['documents'] if d.get('source_format') == 'pdf']
with_text = sum(1 for d in pdfs if d.get('plain_text') and not d['plain_text'].startswith('[PDF -'))
print(f'PDFs with text: {with_text} of {len(pdfs)}')

# Documents without a patient
unlinked = [d for d in b['documents'] if not d.get('patient_id')]
print(f'Documents without a patient: {len(unlinked)}')
for d in unlinked[:5]:
    print(' ', d['source_format'], d['source_file'])

# What stage 6 removed
pre = json.load(open('dashboard_data.prefilter.json', encoding='utf-8'))
print(f"Patients before and after the filter: {len(pre['patients'])} and {len(b['patients'])}")

For problems with the add-on modules, check the log each writes and the Quality control page.