skip to content>_lesfer

Aug 10, 2026 · open-source · in progress

Bab

A self-hosted AI gateway built around governance. Applications get scoped virtual keys instead of provider credentials, and every request passes one gate that decides who may use which model, within which limits and guardrails, and records who spent what.

role Everything — product, architecture, backend, console, and operations

  • Python
  • FastAPI
  • PostgreSQL
  • Redis
  • SQLAlchemy
  • React
  • TypeScript
  • TanStack Query
  • Docker
  • OpenTelemetry

Case study

Contents
  1. 00Top
  2. 01Highlights
  3. 02Overview
  4. 03The Gateway
  5. 04Governance That Composes
  6. 05Proving It Afterwards
  7. 06Tested Without Paying For Inference
  8. 07Why This Project Matters

Every organization that uses AI eventually hits the same question: who is allowed to spend what, through which provider, with which key, and who can prove it afterwards? Bab answers it in one place. It holds the upstream provider credentials, hands applications scoped virtual keys instead, and enforces access, limits and guardrails at a single gate, with every request attributed and traceable. It's also where provider reliability belongs, as infrastructure rather than application code: the argument this site's own copilot makes when it leaves retries and provider quirks to a gateway.

Highlights

  • An OpenAI-compatible gateway for chat, Responses, completions, embeddings, transcription, speech and image generation, plus native Anthropic Messages, and native Gemini upstreams translated to the OpenAI shape. OpenAI, OpenRouter, Mistral, Gemini and Anthropic come built in; custom providers can use any of the three protocols.
  • Encrypted credential storage behind a secret-backend abstraction, with expiration, replacement-based rotation, credential pools, validation, health state, and a preview of the applications a change would affect.
  • An organization, team, project and virtual-key hierarchy with scoped roles, OIDC or local console sign-in, and visible, revocable sessions.
  • Typed limits on requests, tokens, spend, audio duration, characters and generated images, over rolling, calendar, per-request or lifetime windows, enforced hard or in monitor-only mode.
  • Guardrails on requests and responses: deterministic detectors for sensitive terms, PII, secrets, URL domains, tool calls and attachments, and external semantic evaluators (classifier, embedding similarity or an LLM policy) with explicit data egress, confidence thresholds and a chosen failure posture.
  • Signed, hash-chained audit events, versioned governance snapshots without secrets, per-request traces down to the exact upstream credential used, and alerts delivered over signed webhooks.
  • Built and tested against dbab, a deterministic fake provider I wrote alongside it, so provider behavior is exercised in tests without paying for inference.

Overview

Bab is an MIT-licensed alpha: a FastAPI backend and a React administration console, deployed as a single-host PostgreSQL, Redis and TLS Compose topology. Applications never see provider credentials. They hold virtual keys whose access is what every policy above them in the hierarchy allows, and every request they make is bounded, attributed and recorded.

The Gateway

Applications point their existing SDKs at Bab and change nothing else. Before dispatch, Bab works out what the request needs (its operation, modalities, features such as streaming, tools or structured output) and removes routes that can't serve it. Parameters are compiled for the selected model: an output-token limit becomes whatever that model calls it upstream, and a parameter the model doesn't support is rejected rather than silently dropped. Credential selection is a routing decision with priority, health and weight; failures follow explicit retry and fallback rules within per-attempt deadlines; and usage lands in exactly-once projections, which is what makes the spend numbers trustworthy enough to bill against.

Governance That Composes

The control plane's core problem is inheritance: policies have to combine without surprising anyone. The rules are small and strict. Policies at the same scope union their routes; a child scope can narrow what it inherits but never widen it; an archived team or project stops its descendants' traffic while keeping their history. When a hard token or spend limit applies, a request needs a finite output ceiling, and Bab never lowers one the client asked for without saying so. Guardrails follow the same philosophy of rehearsal before enforcement: they can run in monitor or dry-run mode, simulations show what would have been blocked, and impact previews list what a change would touch.

Proving It Afterwards

Governance without evidence is theatre. Administrative actions land in signed, hash-chained audit events that can be verified. Organization configuration is captured in versioned snapshots covering workspace, identity, routing, policy, guardrails and notification channels, without secret material. Usage and spend can be explored and exported per team, project and key. Structured logs, metrics and OpenTelemetry traces come with an opt-in Grafana, Prometheus, Loki and Tempo stack for local work.

Tested Without Paying For Inference

A gateway that only works against real providers is expensive to test and impossible to test for the failures that matter: malformed responses, slow streams, quota errors. So Bab has a companion, dbab: a deterministic, configurable fake provider that speaks the OpenAI-compatible, Anthropic and Gemini APIs, with scripted latency, errors, retries and rate limits per named profile. Every change that touches upstream behavior gets a dbab contract scenario before any paid-provider test.

Why This Project Matters

Bab is infrastructure engineering where it gets hard: correctness under concurrency, security as a data model rather than a checklist, and observability designed in rather than bolted on. It's the unglamorous layer that lets every other AI project stay simple.