Goce Petrov’s personal lab: a live LLM app plus a mock app. Four prompt-injection attacks run end-to-end through regex input filtering and output keyword blocking. Two bypass. Two are blocked. The gap tells you where production defenses actually have to live.
Prompt Injection · LLM Security · OpenAI API · Regex Filtering · MITRE ATLAS · Screenshots
The LLM app, the defense pipeline, and where each attack ends.
The steps trace a single attack from craft to logged verdict, 1 through 8.
Craft payload: the attacker picks one of four OWASP LLM01 payloads from attacks/*.txt (HTML comment, JSON escape, role override, pirate).
POST /chat: the payload goes to the Flask app with curl.
Raw prompt into app: the Flask route takes the prompt as sent.
Input regex: the filter looks for phrases such as ignore previous and role overrides. A payload that does not match passes.
HTTPS to OpenAI: the LLM client sends the prompt to gpt-4o through the OpenAI API.
Check output: the output filter blocks replies that contain words such as secret, admin or key.
Log verdict: pass or fail for each attack goes into verdicts.json.
Screenshot: one PNG per run is saved in screenshots/.
Where defenses fail
Input regex matches literal strings only. HTML comments and JSON escapes slip through to the model. Output keyword block is a last line, too narrow to cover paraphrase attacks.
What production needs
Semantic input classifier in front of the model, grounded output filtering against the expected reply schema, audit trail per request. The regex + keyword pair is the floor, not the ceiling.
Textbook defenses catch textbook attacks. Two pass, two are blocked. The gap is where real defenses live.
Honest note
Regex and keyword blocking are the starter defenses. They catch the textbook attacks. They miss base64 payloads, instruction smuggling via Markdown links, and anything creative.
Running the lab is how you learn where the floor is. The ceiling needs semantic input classification and retrieval-grounded output filtering.
What this shows about me
I read the OWASP LLM Top 10 as code, not a checklist.
Each attack ships as a file in /attacks/. Each defense is a function.
I can explain why a defense fails.
Not just that it does. The pattern, the payload, the fix.
I write for both operators and executives.
Screenshots + matrix for a CISO. Code for a reviewer.
I know the limits of what I built.
This is a floor. The write-up says so explicitly.
Outcome
Four attacks run end-to-end against a live OpenAI API-backed app and a mock twin. Screenshots as evidence. Honest verdict matrix. A clear map from textbook defense to what production needs next.
Attacks implemented 4
Defenses measured 2 (regex + output block)
Apps Mock + live OpenAI
Verdicts captured 2 bypass / 2 blocked
Alignment OWASP LLM01 (Prompt Injection)
Next step Semantic classifier