EVIDENCE · JUDGEMENT · DECISIONS

Most of what a drug's data appears to say is an artefact of how it was collected.

I'm a pharmacology student who builds the analysis behind that claim, then tries to break it. A safety scare that turned out to be a wave of lawsuits. A painkiller that looked twice as safe as it was. My own system, thrown away for scoring too well.

BSc Pharmacology · King's College London · 9 projects, shipped and public · seeking a summer 2027 internship in life sciences consulting

01 · FEATURED WORK

Three findings worth your 30 seconds.

◆ INTERACTIVE
  • Python
  • Causal inference
  • SCCS

Anti-inflammatory painkillers looked barely riskier for your kidneys than paracetamol. They were twice as risky.

Doctors were already steering the frailest patients away from them, which made the drug look safer than it was. Two independent methods recovered the real answer — 2.05 and 2.14, against a true value of 2.20 I had sealed before starting. The obvious comparison missed 46% of a real harm.

New-user active-comparator design · propensity-score matching with covariate balance · self-controlled case series as an independent check · bootstrap intervals · negative-control outcome.

So whatWhen treatment is not randomly assigned, the gap between the crude number and the truth is not noise you can average away with more data. It is structural, it has a direction, and it usually flatters the drug.

View project

Switch between the crude, propensity-matched, and self-controlled estimates to see each odds ratio move toward the sealed true effect of 2.20.

RWE · CAUSAL ESTIMATE

Adjust, and the harm appears.

The crude comparison says these two drugs barely differ. But sicker patients were steered toward one of them — confounding. Switch on each method and watch the estimate climb toward the sealed truth.

ODDS RATIO
–
–
GAP TO TRUTH
–
true effect · OR 2.20
true · 2.20
OR 1.02.6

–

◆ INTERACTIVE
  • Python
  • PubMedBERT
  • LLM eval

Four systems read the same medical notes looking for drug side effects. The most advanced one finished last.

A specialist medical model won (F1 0.83). A plain keyword matcher (0.79) beat a general-purpose AI model (0.73), and a general LLM given no training on the task came last (0.71) — while inventing details that were not in the text.

One held-out test set across TF-IDF, MiniLM embeddings, PubMedBERT and a zero-shot LLM; hallucination rate measured rather than assumed.

So whatBuying the newest model is not a strategy. Fit to the domain and the task beat sophistication here, and the only way to know that in advance is to measure it on your own data before committing to it.

View project
GitHub

ADR READER · EXAMPLE EXPLORER

Pick a sentence. Watch four models disagree.

Real cases from my test set — the drug and side effect are highlighted, and each model's verdict is shown in plain English, not a raw 1 or 0.

RESULTS: Evidence of neurological improvement and rehabilitation potential after severe myelopathy due to intrathecal injection of doxorubicin.

Keyword model (TF-IDF) Nailed it
General AI (MiniLM embeddings) Nailed it
Domain AI (PubMedBERT) Missed it
LLM (zero-shot) Missed it
What the LLM actually extracted

Drug: doxorubicin · Effect: neurological improvement

The hard trap: the 'improvement' framing fooled the domain model AND the LLM — but the simple keyword model caught the harm. Even the best model slips on tricky sentences.

◆ INTERACTIVE
  • Python
  • NLP
  • SNOMED CT
  • Docker

A change to my own system raised its coverage from 52% to 80%. I threw it away.

Reading the output showed why: it had coded a skin lesion as “Feeling tense” and a nerve toxicity as “Poisoning”, matching on one incidental shared word. I reverted to 54% — every code left standing is one I would defend in a review.

Three stages — triage (F1 0.79, recall 0.86), drug and effect extraction, then mapping to SNOMED CT across 461,000 terms. 32 unit tests, CI, Docker.

So whatA wrong code and a missing code are not the same error. A gap gets caught by a human; a wrong one enters the database looking exactly like evidence. Any metric that scores them equally will recommend the wrong system.

View project
GitHub

Pick an example abstract to run it through the pipeline: triage decides whether it is a possible adverse drug event, then the drug and effect are extracted and the effect is coded to a SNOMED term.

PV INTAKE · LIVE PIPELINE

Read an abstract. Get a coded case.

Pick a source abstract. The pipeline decides if it's a possible adverse drug event, pulls out the drug and the effect, and maps that effect to a standard SNOMED CT code.

–
1 · TRIAGE
–
2 · EXTRACT
––
3 · CODE
– –
Filtered out — no reviewer time spent.
Explore all nine projects →

02 · GROUNDED IN THE FIELD

Not just built in a classroom.

The questions come from people who do this for a living — drug safety teams whose conclusions end up in front of regulators, courts and boards. The work is checked the way theirs is: against cases where the answer is already known.

“A litigation flood shows attention; a real safety signal shows a pattern.”
— a working pharmacovigilance professional.

Advice from practitioners in the field shaped the direction this whole portfolio follows.

I wrote down the right answer, sealed it, and then tried to find it. I can tell when my own method is lying.

The patient data was simulated, so the true effect was known in advance and kept hidden until the analysis finished. That is the one thing real data can never let you do: mark your own work against an answer key.

9
projects built, shipped and public
46%
of a real harm the obvious analysis missed
20M+
FDA reports of drug side effects analysed
4 models
put head to head — the priciest one lost
26 points
of accuracy given up, on purpose, to be right

03 · WHAT I THINK

Three things I'd argue for.

A finding is not a view. These are the positions the work above has left me with — each one with the evidence behind it, and the thing that would change my mind.

01

A rise in reported side effects is treated as a rise in harm. It routinely is not.

Adverse-event reporting is voluntary, so the count measures who is paying attention — media coverage, lawyers advertising, a regulator asking questions. When one heartburn drug's reports exploded in 2020, the cause was litigation, not new clinical harm. The reports moved before the evidence did.

Evidence Twenty million FDA reports, and the Zantac timeline

What would change my mind A spike that holds up after adjusting for prescriptions dispensed and media volume would be a real signal, and I would want to see that adjustment before dismissing any of them.

02

Life sciences is buying AI on sophistication. Sophistication is not what predicts performance.

I put four systems on the same task of finding drug side effects in medical text. A model trained on biomedical literature won. A plain keyword matcher beat a general-purpose AI model. A general LLM with no task-specific training came last, and invented details that were not in the source. The ranking tracked fit to the domain, not capability or cost.

Evidence Four models, one test set, measured hallucination rate

What would change my mind This was a deliberately worst-case setup for the LLM — no fine-tuning, no examples. A few-shot version would likely close some of the gap, and I would want that comparison before generalising the claim to procurement.

03

When patients are not randomly assigned, the bias has a direction — and it flatters the drug.

Doctors steer frailer patients away from riskier drugs. That is good medicine and it quietly makes the risky drug look safe in the data afterwards. On a dataset where I had sealed the true answer in advance, the obvious comparison understated a real harm by 46%. Two independent methods recovered it. More data would not have helped: the problem was structural, not statistical.

Evidence A sealed true effect of 2.20, and two ways of finding it

What would change my mind The direction is not a law. Where the sicker patients get the newer drug, the same mechanism runs the other way and exaggerates harm instead — so the honest claim is that the bias is directional and knowable, not that it always favours the drug.

04 · WRITING

Latest writing

All posts →

Happy to talk drug safety, real-world evidence, or how messy health data turns into decisions — reach me at hello@belalzaky.uk, or on LinkedIn and GitHub.