Researcher plants 2m canaries into the AI supply chain. A privacy disaster surfaced.

Test using prompts with unique, trackable secrets proves that data doesn’t stay private.

Researcher plants 2m canaries into the AI supply chain. A privacy disaster surfaced.

TL;DR

  • Researchers planted 2 million digital 'canaries' (trackable secrets) into AI prompts to test data leakage.
  • At least three AI providers were compromised, leading to the theft of sensitive information like AWS access keys.
  • Multiple resellers logged prompts and passed on credentials, exploiting low-price offerings.
  • Several AI providers ignored user-embedded privacy settings such as Zero Data Retention (ZDR) or 'no data collection'.
  • Attackers reused stolen AWS keys extensively to explore cloud resources and gain further access.
  • A Chinese AI lab was found fetching secret URLs from over half the prompts sent to its models, regardless of privacy settings.
  • Hosting AI models on-premises is suggested as the most effective prevention method, followed by direct service from first-tier providers.