tech
December 15, 2025
Creative Commons announces tentative support for AI 'pay-to-crawl' systems
The organization offered support for this idea of an AI marketplace, and suggested several guiding principles.
TL;DR
- Creative Commons (CC) is cautiously supportive of "pay-to-crawl" technology for AI.
- Pay-to-crawl would automate compensation for website content used by AI webcrawlers for training.
- This system could help websites sustain content creation and combat the loss of search traffic due to AI chatbots.
- CC previously announced a framework for open AI ecosystems and dataset sharing.
- Companies like Cloudflare, Microsoft, ProRata.ai, and TollBit are involved in pay-to-crawl development.
- CC suggested principles for responsible pay-to-crawl, including avoiding default settings, allowing throttling, preserving public interest access, and ensuring interoperability.
- CC also supports the Really Simple Licensing (RSL) standard, which dictates crawler access without blocking.
Continue reading
the original article