AI Security Gateway

OpenAI-compatible middleware between applications and any LLM provider. MSc thesis deposited, pending defense.

Marcos Martín
Playground chat routed through the gateway

Point an existing OpenAI-compatible client at the gateway Base URL. Every request goes through a defense-in-depth pipeline before it reaches OpenAI, Anthropic, or a local Ollama model via LiteLLM. No SDK, no client rewrite — security is toggled per request with headers.

Pipeline

PhaseWhat happens
1. Threat (inbound)Sanitisation → length limits → OWASP-style regex → LLM-as-judge ∥ DistilBERT (parallel)
2. DLP (inbound)Bilingual spaCy NER + regex (email, IBAN, phone, person, card, DNI). Reversible tokens.
3. RouterLiteLLM dispatches to the requested model.
4. OutboundTokens restored; outbound DLP can redact leaked secrets / honeytokens.
5. ObservabilityLatency by phase, block reason, estimated cost — async to SQLite.
Per-conversation security controls
Playground: enable or disable Threat, DLP and outbound DLP per request; pick the L4 model and NER entities.
Preconfigured attack cases
Built-in cases document the layer they target and the expected outcome before you run them.

Evaluation

The dashboard runs the case battery without controls vs. a protected profile. Headline result: 17/17 PASS on the protected profile. Unit tests: ~175 pytest.

Evaluation KPI summary
Pass/fail counts, success rate and per-category breakdown.
Case-by-case evaluation detail
Layer hit, HTTP status and latency without vs. with controls.
Operational metrics summary
Operational view: request volume, latency, defensive interventions.

Stack

FastAPI · Pydantic v2 · LiteLLM · spaCy (EN/ES) · Hugging Face Transformers + PyTorch (DistilBERT) · Streamlit · SQLite · Docker.

Honest limits (MVP)

  • No enterprise authentication or authorization in front of the gateway.
  • SQLite is not a SIEM.
  • Regex plus a classifier do not cover every jailbreak.

Write-ups