tech
I Vibe Coded a Security Risk
The app worked. Nobody, including me, had checked whether it was safe.
TL;DR
- The author built an app called Tastemaker to collect writing clips and generate style guides using AI.
- A new feature intended to connect user profiles directly to AI agents to retrieve style guides introduced a security vulnerability.
- Another AI model identified a public registration route that should not have been public, posing a security risk.
- The vulnerability was discovered before any user data was accessed, but the feature was immediately taken down.
- The experience highlights the potential for 'task crossover' and the 'illusion of explanatory depth' when using AI tools, leading to overconfidence in understanding complex systems.
- The author emphasizes the need to understand basic principles of a field, consult human experts, and not solely rely on AI's self-assurance for readiness and safety.