How DINO and SAM are Helping Modernize Essential Medical Triage Practices
By leveraging advanced AI models, teams at the University of Pennsylvania are aiming to bring cutting-edge automation to emergency response.
TL;DR
- Triage, originally a battlefield necessity, is evolving with advancements in AI, computer vision, and robotics.
- DARPA has launched a three-year challenge to spur innovation in autonomous triage systems for mass casualty incidents.
- The challenge requires teams to develop systems that can detect physiological signatures and provide real-time casualty identification and injury assessment.
- Simulated real-world conditions, including darkness, dust, and rubble, are used to test the systems' efficacy.
- The PRONTO team, comprised of experts from Penn Medicine and Penn Engineering, is developing a multi-robot system using Meta's SAM and DINO models.
- PRONTO's system utilizes drones for initial surveying and ground robots for detailed imaging and vital sign capture.
- Advanced AI models like SAM 2, DINO, and Grounding DINO are employed for object segmentation, injury classification, and feature detection.
- The system aims to identify critical patient information such as heart rate, respiration rate, awareness, and the presence of wounds or amputations.
- The DARPA Challenge aims to create a substantial dataset for evaluating mass casualty response strategies and pushing life-saving technology towards deployment.