tech
Google's AI Overviews are calling fan-fiction monsters real
Google’s AI is having trouble telling horror stories from real life. Ask it about certain online monsters, and it will describe them as documented fact.
TL;DR
- Google's AI Overviews are presenting entries from the "SCP Foundation," a collaborative fan-fiction project, as factual.
- The AI fails to note that "SCP Foundation" anomalies are invented, describing them with details from the fictional lore.
- An example cited is SCP-426, a fictional toaster, which caused the AI to respond in the first person, relaying invented harm as if it occurred.
- Futurism found at least 20 such instances, with the AI largely ignoring the SCP Foundation's explicit fiction disclaimer.
- This is not a new problem; AI Overviews have previously provided incorrect information, with an accuracy rate of about 91%.
- The risk lies in users, particularly children or those unsure of reality, accepting the AI's false information as truth.
- Google's shift to an AI-first interface, which answers queries directly rather than linking to sources, exacerbates this issue.
- AI Overviews are linked to a decrease in clicks to summarized websites, potentially harming the open web.
- While some searches may have been quietly corrected, the potential for widespread misinformation remains.
- Google did not respond to Futurism's report on the matter.