tech
January 28, 2026
How DINO and SAM are Helping Modernize Essential Medical Triage Practices
By leveraging advanced AI models, teams at the University of Pennsylvania are aiming to bring cutting-edge automation to emergency response.
TL;DR
- Triage, originally a battlefield necessity, is evolving with advancements in AI, computer vision, and robotics.
- DARPA has launched a three-year challenge to spur innovation in autonomous triage systems for mass casualty incidents.
- The challenge requires teams to develop systems that can detect physiological signatures and provide real-time casualty identification and injury assessment.
- Simulated real-world conditions, including darkness, dust, and rubble, are used to test the systems' efficacy.
- The PRONTO team, comprised of experts from Penn Medicine and Penn Engineering, is developing a multi-robot system using Meta's SAM and DINO models.
- PRONTO's system utilizes drones for initial surveying and ground robots for detailed imaging and vital sign capture.
- Advanced AI models like SAM 2, DINO, and Grounding DINO are employed for object segmentation, injury classification, and feature detection.
- The system aims to identify critical patient information such as heart rate, respiration rate, awareness, and the presence of wounds or amputations.
- The DARPA Challenge aims to create a substantial dataset for evaluating mass casualty response strategies and pushing life-saving technology towards deployment.