Google DR AI stumbled in Thai clinics

Apr 2020

Google’s diabetic retinopathy screener rejected low-light scans and jammed nurse workflows, forcing clinics in Thailand to keep patients waiting despite the promised instant triage.

Incident Details

Perpetrator:Healthcare Pilot

Severity:Facepalm

Blast Radius:Manual re-work, patient suffering, workflow disruption, health and triage impacts.

Tech Stack

Google HealthDiabetic retinopathy modelDeep learningAI screening workflow

References

MIT Technology Review: Google's medical AI was super accurate in a lab ↗Google Research: Human-centered evaluation of the Thai deployment ↗