Work EHR ML Risk Model

A model that flags every patient as high risk catches every case. It is also useless.

Python · scikit-learn · pandas

The finding

The model scored 0.91 AUC, but the number that mattered was where you set the threshold — what it costs to miss a case, weighed against what it costs to raise a false alarm.

Method

Logistic regression and random forest on features derived from synthetic patient records, with deliberate control of temporal data leakage.

So what

Headline accuracy is a marketing number. The operating point is the business decision, and it belongs to whoever carries the cost of each kind of error — not to the person who built the model.

Interactive

Drag the threshold to see how many true hypertension cases the model catches versus how many healthy patients it falsely flags.

EHR ML · CLASSIFICATION THRESHOLD

Where do you draw the line?

The model gives every patient a probability of hypertension. You choose the cut-off for calling it "positive." That choice isn't maths — it's clinical. Drag it.

Flag if probability ≥ 0.50 only flagging near-certain cases

RECALL

–%

of real cases caught

PRECISION

–%

of flags are correct

FALSE ALARMS

–

healthy, flagged

Of 76 patients who truly have hypertension — – caught, – missed

screen aggressively only if certain

Full write-up

What it does

Takes features derived from Synthea patient records and predicts whether a patient has hypertension. I trained and compared two models — logistic regression and random forest — and evaluated them on held-out data.

What I spent the most time on

Three things mattered more than model choice:

Data leakage. EHR data has a temporal dimension that’s easy to ignore when working with CSVs. I had to be deliberate about which features were available before a diagnosis versus after. Getting this wrong produces an optimistic model that could not work in deployment.

Precision/recall trade-off. A model that calls every patient high-risk has perfect recall. I spent time understanding what different probability thresholds actually mean in a clinical framing — what it costs to miss a case versus to flag a false one — even though this is synthetic data with no real stakes.

Synthetic-data performance. Synthea data is generated from statistical distributions, not from real patients. A high AUROC on Synthea says something about the model fitting those distributions; it says very little about whether the model would transfer to real EHR data. I tried to be clear about that distinction throughout.

Stack

Python · scikit-learn · pandas