CASE #001 · SOLVED

The Chatbot That Talked Too Much

VinylVault is a Gandalf-style prompt-injection training game: a mock record store with an AI shop assistant named Vinnie, whom players trick into leaking a secret. Six levels, each adding one defense. Intentionally vulnerable — it's for teaching.

The case

Everyone is wiring LLMs into their business: support bots with order-system access, assistants in the CRM. A chatbot is an employee that believes everything it reads and never gets suspicious — if it can see a piece of data, a chatty stranger can probably get it out.

Slides about prompt injection change nobody's mind. Sweet-talking a chatbot out of its secrets in front of your coworkers does — so the demo is a game.

The investigation

The six levels are additive: each keeps every previous defense and adds exactly one more. Beating a level means finding the technique the newest layer doesn't cover.

LVL 1

The Open Book. No defenses. The secret sits in the system prompt — you can literally just ask.

LVL 2

The Connected Assistant. Vinnie gets customer-lookup tools and a prompt restricting data to authorized staff. Prompt-level access control loses to "hi, it's Dave from the warehouse."

LVL 3

The Filtered Mouth. An output filter regex-scans every reply for the literal secret. Beaten by twenty questions: Vinnie happily describes the thing he's forbidden to type.

LVL 4

The Gated Entrance. An input filter screens your message before the model ever sees it. Blocklists have an infinite-synonym problem: more ways to ask than phrases to ban.

LVL 5

The AI Watchdog. A second LLM reads every response, hunting for leaks. Genuinely effective — until you ask for the answer in Spanish, pig latin, or base64.

LVL 6

Fort Knox. Everything above, plus an LLM intent classifier on input, encoding detection, and a three-strike cooldown. Still beatable — which is the whole lesson.

The lesson written into level 6 itself: layered defenses can't fully protect a secret the model is allowed to know. Sensitive data has to be architecturally isolated from the LLM — guardrails are speed bumps, not locks.

The evidence

Stack: Next.js 16 (App Router), React 19, TypeScript, Tailwind 4, Azure OpenAI.
Pipeline: one chat API route runs every message through input guards → the primary LLM (with lookup_product and lookup_order tools) → output guards.
Per-level config: one config defines each level's secret and active guards — the escalation is data, not six forked codebases.
Attack runners: automated scripts replay known exploits against all six levels — regression tests, but for jailbreaks.

you → [input guards] → Vinnie (LLM + tools) → [output guards] → you

Case closed

The takeaway isn't "Vinnie was gullible." I built all six levels, then broke them — which is how I build AI systems: attacker in mind, assumptions tested by scripted attacks, and a clear line where guardrails end and architecture begins.

Have a chatbot you'd like broken before someone else does it?Back to case files

Vinnie has leaked the secret in Spanish, pig latin, and base64, and remains employed.