Skip to content

How the parts fit together

Explanation
  • operator
  • evaluator

Visual companion to PRD v0.2. This document renders the v1 architecture as a Mermaid diagram and walks through the architectural choices for contributors and procurement evaluators. The diagram below covers the full M1–M4 target architecture with M5+ forward-looking elements drawn as dashed slots; what runs at the M1 baseline is a strict subset, called out in the milestone breakdown further down.

For a per-capability shipped-vs-deferred catalog with verification paths, see HONEST-STATE.md.

A browser, the Word add-in, or a Slack/Teams bridge sends requests to the API, which manages matters, chats, skills, and other records. When a request needs a model, the API calls the Inference Gateway — the single service that selects a provider, applies tier and privacy rules, and holds every privileged provider API key. Storage (PostgreSQL, Redis, RustFS/S3-compatible) lives entirely inside the operator’s environment; two background worker processes (ingest-worker for document processing, arq-worker for Playbook, Tabular Review, and Autonomous Layer jobs) do the async work. The Word add-in and the Slack/Teams bridge are shipped as plumbing, with parts of their feature surface still deferred — a box in the diagram is not proof a workflow is finished. Mn tags describe milestones, not guarantees; HONEST-STATE.md is the authoritative shipped-vs-deferred record.


When you ask a question, the app sends it to the application API. If you attach a knowledge base, the system retrieves relevant passages before asking a model. The model request goes through the Inference Gateway, which applies the configured routing and privacy rules.

How a question reaches a model
How a question reaches a model How a question reaches a model

The app sends the question and chosen context to the application API. The API assembles the request using document storage and retrieval, then sends model requests through the Inference Gateway. The configured route selects a local or cloud model; these are alternative routes. The answer returns through the gateway and API to the app.

Open full-size diagramOpen full-size diagram
Diagram source
flowchart TB
    accTitle: How a question reaches a model
    accDescr: The app sends the question and chosen context to the application API. The API assembles the request using document storage and retrieval, then sends model requests through the Inference Gateway. The configured route selects a local or cloud model; these are alternative routes. The answer returns through the gateway and API to the app.
    app["LQ.AI app"] -->|Question and context| api["Application API"]
    records["`Documents
and records`"] <-->|Read and store| api
    api -->|Assembled request| gateway["Inference Gateway"]
    gateway -->|Local route| local["Local model"]
    gateway -->|Cloud route| cloud["Cloud model"]

Document storage and retrieval support this flow within the deployment. The document-search model can have a different provider from the chat model: choosing local chat does not by itself make document search local. Other enabled connections also need their own review. See what information leaves the deployment for the full accounting.

This is a simplified request flow. The full system diagram includes storage, workers, optional integrations, and capability limits.

flowchart TB
%% =========================================
%% CLIENT SURFACES
%% =========================================
subgraph CLIENTS["💻 Client Surfaces"]
direction LR
web["<b>Web App</b><br/>OpenWebUI fork<br/><i>M1</i>"]
word["<b>Word Add-In</b><br/>Office.js<br/><i>M3</i>"]
slack["<b>Slack/Teams<br/>Bridge</b><br/>profile-gated<br/><i>M3 optional</i>"]
today["<b>Today View</b><br/>workflow concierge<br/><i>M5+ forward-looking</i>"]:::forwardLooking
end
%% =========================================
%% BACKEND (FastAPI)
%% =========================================
subgraph BACKEND["🧠 LQ.AI Backend (FastAPI)"]
direction TB
api["<b>OpenAPI 3.1 Surface</b><br/>Authn/Authz · RBAC · MFA option<br/>Privilege-aware Audit Log<br/><i>M1</i>"]
subgraph CORE_SERVICES["Core Services"]
direction LR
proj["<b>Project Service</b><br/>matter-scoped<br/>containers<br/>privilege flag<br/><i>M1</i>"]
skill["<b>Skill Service</b><br/>incl. Org Profile<br/>singleton<br/><i>M1</i>"]
playbook["<b>Playbook + Tabular Service</b><br/>LangGraph workflows:<br/>playbook execute · Easy Playbook ·<br/>tabular multi-doc review<br/><i>M3</i>"]
doc["<b>Document Pipeline</b><br/>PyMuPDF +<br/>Citation Engine<br/>(OCR deferred — DE-320)<br/><i>M2</i>"]
knowledge["<b>Knowledge Service</b><br/>pgvector + FTS;<br/>contract-relationship<br/>graph slot (not built)<br/><i>M1</i>"]
autonomous["<b>Autonomous Layer</b><br/>five-phase executor · cron ·<br/>watches · schedules ·<br/>per-user memory · precedent board<br/><i>runs as arq job on arq-worker</i><br/><i>M4</i>"]
end
worker["<b>Async Workers</b><br/>arq-worker — Easy Playbook · Tabular Review ·<br/>Autonomous sessions + cron (arq:m3a6 queue)<br/>ingest-worker — document pipeline (ingest queue)<br/><i>M1 ingest · M3 arq-worker · M4 autonomous</i>"]
mcp["<b>MCP-Client Subsystem</b><br/>integration substrate;<br/>architectural slot opened in M2<br/>even though no connectors ship until M5+<br/><i>M2 slot</i>"]
signal["<b>Signal Aggregation Service</b><br/>workspace events<br/>via MCP<br/><i>M5+ forward-looking</i>"]:::forwardLooking
end
%% =========================================
%% INFERENCE GATEWAY
%% =========================================
subgraph GATEWAY["🚪 Inference Gateway (~3,000 LOC, built in-house)"]
gateway["<b>Single security-bounded service</b><br/><br/>Auth → Router → Rate Limit → <b>Tier Derivation</b> →<br/><b>Anonymization (M2)</b> → Provider Adapters →<br/>Cost Tracker → Telemetry<br/><br/>Annotates every request with <i>routed_inference_tier</i> (1–5)<br/>Refuses requests below skill or Project minimum tier<br/><br/><i>M1 baseline; Anonymization adds in M2</i>"]
end
%% =========================================
%% STORAGE (operator's environment)
%% =========================================
subgraph STORAGE["💾 Storage (operator's environment)"]
direction LR
postgres["<b>PostgreSQL + pgvector</b><br/>app data · vectors · FTS<br/>privilege-aware audit log<br/>work-product attribution"]
redis["<b>Redis</b><br/>sessions · queues<br/>pubsub"]
objectstore["<b>RustFS / S3-compatible</b><br/>file storage"]
end
%% =========================================
%% LLM PROVIDERS (operator's choice)
%% =========================================
subgraph PROVIDERS["🌐 LLM Providers (operator's choice; configured per-deployment)"]
direction LR
cloud["<b>Cloud Providers</b><br/>Anthropic · OpenAI · Azure OpenAI<br/><i>Vertex · Bedrock deferred — DE-034/035</i><br/><i>Tier 2–4 depending on configuration</i>"]
local["<b>Local Providers</b><br/>Ollama · vLLM · llama.cpp<br/>any OpenAI-compatible endpoint<br/><i>Tier 1 — air-gap-capable</i>"]
end
%% =========================================
%% EXTERNAL (M5+ via MCP)
%% =========================================
subgraph EXTERNAL["🔮 External Workflow Sources via MCP — M5+ forward-looking"]
direction LR
email["<b>Email</b><br/>Gmail · Outlook"]:::forwardLooking
calendar["<b>Calendar</b><br/>Google · Microsoft"]:::forwardLooking
tasks["<b>Task Systems</b><br/>Asana · Linear ·<br/>Jira · Monday"]:::forwardLooking
crm["<b>CRM</b><br/>Salesforce ·<br/>HubSpot"]:::forwardLooking
docstore["<b>Doc Stores</b><br/>SharePoint · Drive ·<br/>OneDrive · Box"]:::forwardLooking
end
%% =========================================
%% OBSERVABILITY (optional)
%% =========================================
subgraph OBS["📊 Observability (optional, off by default)"]
direction LR
otel["<b>OpenTelemetry</b><br/>→ Grafana / Tempo / Loki<br/>→ Honeycomb · Datadog · etc."]
langfuse["<b>Langfuse</b><br/>self-hosted<br/>LLM-specific tracing"]
end
%% =========================================
%% EDGES — client to backend
%% =========================================
web -->|"OpenAPI 3.1<br/>over HTTPS"| api
word -->|"OpenAPI 3.1<br/>over HTTPS"| api
slack -->|"OpenAPI 3.1<br/>over HTTPS"| api
today -.->|"OpenAPI 3.1"| api
%% =========================================
%% EDGES — backend internal
%% =========================================
api --> proj
api --> skill
api --> playbook
api --> doc
api --> knowledge
api --> autonomous
autonomous -.->|"runs as<br/>arq job"| worker
%% =========================================
%% EDGES — services to gateway
%% =========================================
skill --> gateway
playbook --> gateway
doc --> gateway
knowledge --> gateway
autonomous --> gateway
%% =========================================
%% EDGES — services to MCP slot
%% =========================================
skill -.->|"M5+:<br/>tools<br/>over MCP"| mcp
autonomous -.-> mcp
signal -.-> mcp
%% =========================================
%% EDGES — services to storage
%% =========================================
proj --> postgres
skill --> postgres
playbook --> postgres
knowledge --> postgres
autonomous --> postgres
api --> redis
doc --> objectstore
api -->|audit log| postgres
%% =========================================
%% EDGES — async workers (M1 ingest · M3 arq)
%% =========================================
redis -->|"dequeue jobs"| worker
worker -->|"inference"| gateway
worker -->|"results / chunks"| postgres
worker --> objectstore
%% =========================================
%% EDGES — gateway to providers
%% =========================================
gateway -->|"Tier 2-4<br/>HTTPS"| cloud
gateway -->|"Tier 1<br/>local socket"| local
%% =========================================
%% EDGES — observability
%% =========================================
gateway -.-> otel
gateway -.-> langfuse
api -.-> otel
%% =========================================
%% EDGES — MCP to external (forward)
%% =========================================
mcp -.->|"M5+"| email
mcp -.->|"M5+"| calendar
mcp -.->|"M5+"| tasks
mcp -.->|"M5+"| crm
mcp -.->|"M5+"| docstore
%% =========================================
%% STYLING
%% =========================================
classDef forwardLooking stroke-dasharray: 5 5,fill:#fff8dc,color:#7a5c00,stroke:#c9b037
style CLIENTS fill:#e3f2fd,stroke:#1976d2
style BACKEND fill:#f3e5f5,stroke:#7b1fa2
style CORE_SERVICES fill:#faf5fc,stroke:#9c27b0,stroke-dasharray: 0
style GATEWAY fill:#e8f5e9,stroke:#2e7d32
style STORAGE fill:#fff3e0,stroke:#ef6c00
style PROVIDERS fill:#fce4ec,stroke:#c2185b
style EXTERNAL fill:#fffde7,stroke:#c9b037,stroke-dasharray: 5 5
style OBS fill:#f5f5f5,stroke:#616161

  • Solid borders / saturated fills — components that ship in the v1 release path (M1–M4).
  • Dashed borders / muted fills — forward-looking M5+ elements; the architectural slot is committed in M1–M2 (the MCP-Client Subsystem) so these can be implemented incrementally without core refactoring, but they are explicitly not committed for delivery.
  • Solid edges — request paths that exist at v1.
  • Dashed edges — request paths that exist only when forward-looking elements are present.
  • Mn tags — the milestone at which the component lands.

The architecture is shaped by three commitments:

Self-hosted, with the customer’s keys, in the customer’s environment. Storage (Postgres + Redis + RustFS/S3) lives entirely in the operator’s environment; the application services run alongside. Customer content leaves the operator’s environment only through the Inference Gateway — the inference call to the configured provider and, where an operator has enabled them, gateway-brokered tool calls (case-law lookup, MCP connectors). The gateway is the routing boundary for content, not a stack-wide egress firewall: a few other paths need network access and should be reviewed on their own before a deployment is described as offline — image builds; the ingest worker’s first-run download of Docling’s models from Hugging Face (ingest-hf-cache in docker-compose.yml; the pass is still shipped pending its removal under ADR 0026, and LQ_AI_DOCLING_ENABLED=false skips it in air-gapped environments); optional SMTP email for autonomous notifications (smtp_host, off by default; carries counts, IDs and a receipt link, not content); and OpenTelemetry export (OTEL_EXPORTER_OTLP_ENDPOINT, off by default). There is no LegalQuants-side SaaS holding customer data — by design, no telemetry is emitted by default, and what telemetry the operator opts into contains no content. (See PRD §5.7 No Telemetry by Default.)

The Inference Gateway is the security boundary. It is the single component holding privileged provider API keys and the only egress path for customer prompts. We build it in-house in ~3,000 lines of focused Python rather than adopting LiteLLM (whose security history includes proxy auth bypasses and SSRF in document loaders) — for an open-source project where users may run with our defaults, that surface area is unacceptable. Every request passes through Auth → Router → Rate Limit → Tier Derivation → Anonymization (M2) → Provider Adapters → Cost Tracker → Telemetry, and is annotated with the derived Inference Tier (1–5) before leaving. Skills and Projects can refuse to run below their declared minimum tier. (See PRD §4 The LQ.AI Inference Gateway and §4.7 Anonymization Layer.)

The architectural slot for M5+ is opened in M1–M2. The MCP-Client Subsystem (per PRD §8 M1 deliverables) is committed in the v1 architecture even though no connectors ship before M5+. The cost is small (~2 weeks of architectural work and documentation); the value is significant (the M5+ Forward-Looking Workflow Intelligence direction does not require retrofitting MCP into a v1 architecture that did not anticipate it; community contributors can plug in MCP servers — for email, calendar, task systems, document stores, CRM — without core changes). The Signal Aggregation Service is shown as a forward-looking dashed component because it is the next layer above the MCP-Client substrate; it lands at M5 in the forward-looking roadmap.


The diagram is the M1–M4 target architecture; the milestone-by-milestone breakdown shows what is added at each release.

M1 is the foundation release: a self-hostable deployment that delivers conversational legal AI on top of the ten starter skills, with the engineering surfaces that M2–M4 capability work builds on. The surfaces below are wired end-to-end and covered by Cypress E2E; for the authoritative per-capability shipped-vs-deferred catalog with verification paths, see HONEST-STATE.md §§1–6.

User-facing surfaces that ship in M1:

  • Conversational core: multi-turn chat with persistent history, full-text search across chat history, streaming SSE responses, matter (project) workspaces at /lq-ai/matters/[id] with inherited files/skills/knowledge bases.
  • Skill library and Skill Creator: browse built-in, user, and team skills; capture a chat reply as a skill; wizard-based authoring from scratch; fork built-in or team skills; skill versions tab with per-version audit; Try-It sandbox per skill. Slash-invoked skills render a provenance pill on every response.
  • Knowledge bases: create knowledge bases, attach documents, upload PDFs, ingest to ready state (pgvector + FTS hybrid retrieval; full Citation Engine pipeline lands M2).
  • Saved Prompts: standalone library surface with one-click “Use in chat.”
  • Receipts drawer: per-event provenance drawer in chat, including Wave 7.2 source enrichment. Available on every chat message.
  • Enhance Prompt (⌘E): prompt-expansion skill available per-chat.
  • Tier-floor refusal + admin override: tier enforcement UI with admin-only override for privileged matters.
  • Audit log and admin viewer: all sensitive actions logged; admin surface at /lq-ai/admin/audit-log.
  • GDPR-aligned export and account deletion: per-user data export and account deletion flows.

Engineering infrastructure that ships in M1:

  • Web App (OpenWebUI fork) and FastAPI backend.
  • Project Service (matter-scoped containers, including the optional privileged: true flag).
  • Skill Service, including the Organization Profile as a singleton skill.
  • Knowledge Service (pgvector + FTS hybrid retrieval, baseline; contract-relationship graph follows in M4).
  • Inference Gateway core: Auth, Router, Rate Limit, Tier Derivation, Provider Adapters (Anthropic, OpenAI, Ollama), Cost Tracker, Telemetry.
  • Privilege-aware Audit Log fields (privilege_marked, privilege_basis, routed_inference_tier).
  • MFA-mandatory option and session-timeout defaults.
  • Compliance Alignment Pack documentation (docs/compliance/).
  • Code & Supply-Chain Transparency artifacts (SBOM, signed images, SLSA-3 build provenance, public threat model).
  • Cypress E2E: 6 LQ.AI specs (wave-a-chrome, wave-b-surfaces, wave-c-matters, wave-d1-power-features, wave-d2-skill-creator, wave-m1-final-surfaces) covering all user-facing surfaces listed above.

Not in M1 (shipped in M2):

  • Citation Engine verification pipeline — shipped in M2-A through M2-D as a 4-stage cascade (exact match → tolerant match → paraphrase judge → ensemble). See PRD §3.3 and HONEST-STATE.md §3.
  • Anonymization Layer middleware — shipped in M2-B3 with M2-D2 retrieval-context skip + M2-D3 privileged-project handling. See PRD §4.7 and the honest validation-posture caveat at docs/security/anonymization.md.
  • MCP-Client Subsystem architectural slot — moved to M2 during M1 planning to keep M1 scope focused; M2 retained the slot as a design commitment with no connectors shipping until M5+.

Storage and providers stand up in M1; both deployment modes (Mode 1 cloud, Mode 2 local) are operational.

M2 — Citation Engine, Anonymization, and Azure OpenAI (SHIPPED 2026-05-17)

Section titled “M2 — Citation Engine, Anonymization, and Azure OpenAI (SHIPPED 2026-05-17)”

Shipped:

  • Citation Engine — 4-stage verification cascade (exact_match / tolerant_match / paraphrase_judge / ensemble_strict | ensemble_majority) with M2-C2 UI rendering states, M2-D1 ensemble + tier envelope, M2-E2 per-judge cost calibration. See PRD §3.3.
  • Anonymization Layer — pre/post middleware in the Inference Gateway pseudonymizing sensitive entities before the model call and rehydrating on response. Retrieval-context skip (M2-D2) leaves source quotes intact for citation grounding. Privileged-project handling (M2-D3) cross-cuts with tier-floor enforcement. Honest validation posture: custom recognizers + integration are tested; Presidio default-recognizer recall/precision on legal corpus is empirically unmeasured (DE-282 invites community contribution).
  • Multi-Model Ensemble Verification — landed as Stage 4 of the Citation Engine cascade rather than as a full-chat-completion surface. The privacy posture surfaces as the maximum tier across the ensemble (the weakest envelope), persisted on message_citations.tier_envelope.
  • Azure OpenAI provider adapter — M2-E1 (DE-267 closed). Mirrors the OpenAI wire format with deployment-scoped URL + api-key auth + required api_version. Azure AD path (managed identity / service principal) deferred to DE-278.
  • MCP-Client Subsystem architectural slot — retained as a design commitment; no connectors ship.

Not in M2 (intentional, tracked):

  • Side-panel PDF.js viewer with bbox highlighting — UI surface deferred; the citation rendering states (M2-C2) ship the visual contract without the viewer drilldown.
  • “Ensemble runs the full chat path with N parallel reconciled completions” — ensemble landed scoped to Citation Engine verification only; full-chat-ensemble would be a substantial follow-on if it lands at all.

M3 — Playbooks, Word Add-In, Tabular Review, Slack/Teams

Section titled “M3 — Playbooks, Word Add-In, Tabular Review, Slack/Teams”

Adds (all shipped in M3; fresh-install-verified at M3-E1):

  • Playbook Service with LangGraph executor and the Easy Playbook auto-generation wizard. Per-position assessments carry verbatim matched_text + cited_chunk_ids (lexical-FTS anchoring), not the M2 Citation Engine verification cascade — Citation Engine integration for the playbook/tabular executors is deferred. See docs/playbooks.md.
  • Word Add-In plumbing — scaffold + OAuth (M3-B2) + self-served JS bundle + version handshake (M3-B8). The task pane is installable via the unsigned-manifest path (Microsoft 365 Admin Center) and authenticated against the operator’s deployment. The in-Word feature surface inside each tab (chat against the open document, skills with tracked changes, playbook execution, Inference Tier badge) is deferred to M4 / community per PRD §9 DE-287; the signed manifest + enterprise distribution package (M3-B7) is descoped to a community-led effort per PRD §9 DE-295 — only the unsigned-manifest sideload path ships at v0.3.0. See docs/word-addin.md.
  • Tabular Review surface and output_format: table skill mode (M3-C1/C2) + XLSX/CSV export (M3-C4a). Per-cell citations are navigable: the read-side resolves each cell’s grounding chunk to source_file_id/source_page/source_text (post-v0.4.0, #125), though the executor still does not mint real Citation-Engine rows (DE-309); per-column ensemble verification is now honored at execution (#127). See docs/tabular-review.md.
  • Slack/Teams Light Intake Bridge plumbing — slack-bridge + teams-bridge services with OAuth install + admin UI shell. The /lq slash-command surface is deferred to M4 / community per PRD §9 DE-288. The plumbing was verified in isolation only — a real OAuth round-trip against a public-URL tunnel has not been exercised (DE-312, P1). See docs/intake-bridges.md.
  • Second arq worker process (arq-worker) for long-running Playbook + Tabular jobs (the arq:m3a6 queue) — the Easy Playbook generation and tabular extraction pipelines run there to keep within the PRD §3.7 <10-minute NFR without competing with document ingest jobs on the existing ingest-worker queue.

Word add-in delivery flow. The Word add-in’s static bundle (taskpane.html + JS + CSS + manifest icons) lives at word-addin/ and webpacks into web/static/word-addin/. The web container’s SvelteKit build picks the directory up via the existing COPY . . step, so the bundle is served at {deployment_origin}/word-addin/... without any new routing logic. Operators generate per-deployment manifests via the admin UI (/lq-ai/admin/word-addin → GET /api/v1/admin/word-addin/manifest), which templates the operator’s deployment URL into the XML, then sideload the unsigned manifest via Microsoft 365 Admin Center (which warns about the unsigned add-in — expected at v0.3.0). M3-B8 adds a version-handshake endpoint (GET /api/v1/word-addin/version) so the task pane surfaces a clear “update needed” overlay when the installed add-in version drifts from the deployment. The signed manifest + versioned distribution asset (M3-B7) is descoped to a community-led effort once code-signing-certificate procurement closes — see PRD §9 DE-295.

Adds (shipped):

  • Autonomous Layer — a background autonomous executor that runs as an arq job (autonomous_session_job) on the existing arq-worker, not as a separate service. It is a five-phase LangGraph state machine (intake → analysis → drafting → ethics_review → delivery); every external action routes through a single guarded_tool_call chokepoint that enforces three brakes — R4 (per-session / per-trigger cost cap), R5 (external halt + idle watchdog), R6 (phase-gated tool grants). It ships the four primitives — watches, schedules, per-user memory, precedent board — plus honest per-session receipts, per-user opt-in, and a web dashboard. Two cron jobs also run on arq-worker: autonomous_idle_watchdog (reaps idle sessions) and autonomous_schedule_dispatcher (fires scheduled triggers). All of this shares Redis with the API on the arq:m3a6 queue alongside the Easy Playbook + Tabular jobs. See HONEST-STATE.md §5 for the per-capability catalog.
  • M4 design explicitly accommodates multi-step agents with human-in-the-loop guardrails — anticipating the M5+ Workflow Intelligence direction.

Not in M4 (tracked):

  • Contract Repository auto-relationship graph (amendments, restatements, references, master/sub edges over a Knowledge Base) — a separate M4-roadmap capability that is not built; the Knowledge Service’s contract-relationship graph slot remains unfilled.

M5–M7 — Forward-Looking Workflow Intelligence (community-driven; not committed)

Section titled “M5–M7 — Forward-Looking Workflow Intelligence (community-driven; not committed)”

The dashed elements in the diagram show what the M5+ direction adds:

  • Today View as a new client surface alongside the Web App and Word Add-In.
  • Signal Aggregation Service as a backend service consuming workspace events from MCP-connected sources.
  • MCP connectors for email, calendar, task systems, CRM, and document stores — most as community-contributed MCP servers consumed through the existing MCP-Client Subsystem.
  • The Prioritization Engine is a LangGraph workflow inside the Playbook Service; not shown as a separate node because it operates as a skill in the existing substrate.

This direction is not a v1 commitment. It is named in the PRD and reflected in the architecture so that v1 design choices leave room for it. (See PRD §8.5 M5–M7 Forward-Looking Workflow Intelligence and the Workflow Intelligence subsection of §9 for ~14 deferred enhancements bounded to enable community contribution.)


The gateway is shown as a single service in the main diagram; its internal request flow matters enough to call out separately. Every inbound request follows this pipeline:

The Inference Gateway pipeline
The Inference Gateway pipeline The Inference Gateway pipeline

Every inbound request passes through Auth, Router, Rate Limit, Tier Derivation and Anonymization (pre) to the Provider Adapter. The response then passes through Anonymization (post), Cost Tracker and Telemetry before it returns as the outbound response.

Open full-size diagramOpen full-size diagram
Diagram source
flowchart TB
    accTitle: The Inference Gateway pipeline
    accDescr: Every inbound request passes through Auth, Router, Rate Limit, Tier Derivation and Anonymization (pre) to the Provider Adapter. The response then passes through Anonymization (post), Cost Tracker and Telemetry before it returns as the outbound response.
    inbound(["Inbound request"]) --> auth
    auth["`**Auth**
API key resolution; rejects unauthenticated`"] --> router
    router["`**Router**
provider/model selection; fallback chains`"] --> ratelimit
    ratelimit["`**Rate Limit**
Redis token bucket; per-key, per-model`"] --> tier
    tier["`**Tier Derivation**
annotates request with routed_inference_tier 1–5; refuses if below skill or Project minimum`"] --> anonpre
    anonpre["`**Anonymization — pre**
M2; pseudonymizes sensitive entities; stable mapping for the request lifetime`"] --> adapter
    adapter["`**Provider Adapter**
HTTP/gRPC to Anthropic / OpenAI / Azure / Ollama / vLLM; Vertex + Bedrock deferred — DE-034/035`"] -->|response| anonpost
    anonpost["`**Anonymization — post**
M2; rehydrates pseudonyms in response and inside cited chunks; mapping discarded`"] --> cost
    cost["`**Cost Tracker**
tokens × per-model rates; tagged for analytics`"] --> telemetry
    telemetry["`**Telemetry**
OTel traces; Langfuse if configured`"] --> outbound(["Outbound response"])

The pipeline is what makes the Inference Tier model operationally real. The Tier Derivation stage is the choke point: every request gets classified, every classification is logged in the audit trail, and every UI surface reflects the actual routed tier in real time. The user does not have to take the application’s word for it — they can verify the tier badge against the operator’s gateway configuration and against the audit log entries.

The Anonymization stages bracket the provider adapter; pseudonyms exist in the mapping table for the duration of the request (in process memory only — never persisted) and are rehydrated on the way back so the Citation Engine sees the original text for verification. Privilege-flagged Projects disable anonymization by default — for privileged content, the operator is better served by Tier 1 (local inference, no third-party touch) than by an anonymization layer that adds processing steps complicating a privilege analysis.

The provider adapters resolve their keys from the gateway’s secret store, and that store can be administered at runtime: an admin-only /admin/v1/provider-keys surface (proxied by the backend at /api/v1/admin/provider-keys) sets a key Fernet-encrypted into gateway.yaml and hot-applies it to the live adapter with no restart — keys can be added, rotated, and revoked without a redeploy (post-v0.4.0, #128; requires LQ_AI_GATEWAY_MASTER_KEY). Environment-supplied keys continue to work; the listing masks every key and reports its source (env vs runtime).

For the full gateway specification including the configuration YAML and the OpenAPI surface, see PRD §4.


A frequent procurement question: where does customer data actually live? The answer, mapped to the diagram:

User identity / accounts

Where it lives
PostgreSQL
Departs the operator’s environment?
No

Chat history (prompts, responses, citations)

Where it lives
PostgreSQL
Departs the operator’s environment?
No (but the prompt content travels through the Inference Gateway to the configured provider; see below)

Files (uploaded documents)

Where it lives
The configured object store — the bundled RustFS by default, or whatever S3-compatible endpoint S3_ENDPOINT_URL points at
Departs the operator’s environment?
The original bytes: no, provided the object store itself is local. The extracted text is a separate question: attached-file and retrieved-KB text is injected into inference requests verbatim (marked to skip anonymization, so the model sees intact source text) and so departs per the inference-payload row below

Document chunks (post-ingestion)

Where it lives
PostgreSQL (pgvector + FTS)
Departs the operator’s environment?
The stored chunks: no. Computing their vectors sends chunk text to whichever provider backs the gateway’s embedding alias — openai-prod/text-embedding-3-small in gateway.yaml.example — independently of the chat model selected, so a local chat model does not by itself keep embedding local

Skills, Playbooks, Organization Profile

Where it lives
PostgreSQL
Departs the operator’s environment?
No

Project context documents

Where it lives
PostgreSQL
Departs the operator’s environment?
No

Audit log (including privilege_marked, routed_inference_tier)

Where it lives
PostgreSQL
Departs the operator’s environment?
No (unless operator streams to SIEM via syslog/webhook)

Inference request payloads

Where it lives
Travel through the Inference Gateway to the configured provider
Departs the operator’s environment?
Yes, per the routed Inference Tier — see PRD §1.5.2 for what each tier is intended to represent about provider-side retention and training. The tier is a classification the operator declares per provider in gateway.yaml (tier: 4 for the standard commercial API, tier: 3 if a ZDR agreement is in place); the gateway enforces that classification, it does not verify the provider’s actual agreement, account plan or region

Sub-processor mapping table for Anonymization (M2)

Where it lives
In-process only (gateway memory); never persisted
Departs the operator’s environment?
No

OpenTelemetry traces

Where it lives
Operator-configured sink (default OTLP/HTTP to operator’s choice)
Departs the operator’s environment?
Only if the operator points OTel at an external sink

Workflow signals (M5+, dashed)

Where it lives
Postgres via Signal Aggregation Service; sourced via MCP from operator-configured external systems
Departs the operator’s environment?
The MCP servers are operator-configured; signal data leaves the operator’s environment only via the operator’s choice

No data goes to LegalQuants by default. No data goes to the project maintainers. The deployment emits no telemetry to LegalQuants by default; opt-in anonymous usage statistics (version, deployment mode, capability counts) contain no content and are clearly flagged. The audit log can be streamed to the operator’s SIEM via syslog or HTTP webhook for centralized monitoring.

The principal exception — the inference request to the configured provider (and, where enabled, gateway-brokered tool calls; the non-content network paths are listed under What sits where, and why) — is the central security trade-off the Inference Tier model makes explicit. The five tiers (Tier 1 local through Tier 5 consumer/free) capture the spectrum from “no data leaves the deployment” (air-gap, Mode 2) through “data leaves under the operator’s cloud-provider DPA” (Tier 2, customer-hosted Vertex/Bedrock/Azure under operator’s account) through “data leaves under enterprise managed inference with ZDR/no-training” (Tier 3, recommended for most pragmatic enterprise deployments) and beyond.

The application surfaces the routed tier in the chat UI in real time. Skills and Projects can require minimum tiers. Deployments can disallow tiers globally. Every routing decision is in the audit log. Procurement teams can verify from the audit log that requests were routed at the tier the operator configured; whether that provider’s contract, account plan and hosting region actually deliver the retention and no-training terms the tier describes is a check against the provider agreement, which the tier badge does not perform.


A few elements are intentionally omitted from the main diagram for readability; they are present in the architecture and described in the PRD:

  • Reverse proxy (Caddy, Traefik, nginx) sits in front of the Web App and the API for production deployments. See PRD §6.3 Production Deployment.
  • The Web App box holds a second backend. The web image is an OpenWebUI fork, and OpenWebUI ships its own embedded Python backend inside that same image — separate from LQ.AI’s api service — serving /api/config, the auth shell, and the websockets the SPA needs on boot; LQ.AI’s own workspace calls go to the separate api service instead (see web/docs/frontend-dev.md). Both backends mount an /api/v1: the release stack’s bundled proxy resolves the collision by giving LQ.AI’s api its own /lq-prefixed origin and stripping that prefix back before forwarding (docker-compose.release.yml). Keep the two sets of URLs, sign-in paths, and proxy routes distinct when debugging the app or exposing it on another host — see Reverse proxy and TLS.
  • Internal observability between components — every service emits OpenTelemetry traces; the diagram shows the OTel sink rather than every individual instrumentation point. W3C traceparent context propagates across the full api → gateway → provider chain (httpx auto-instrumentation injects on the way out, FastAPI auto-instrumentation extracts on the way in), so a single chat-send surfaces as one end-to-end trace, not three — verified by test_trace_propagation.py in both api/ and gateway/. Beyond HTTP auto-instrumentation, the services emit domain spans for the high-value operations — citation.verify (with per-stage children + short-circuit/budget-fallback events), anonymization.pre/anonymization.post, skill.execute (one per applied skill), inference.dispatch (carrying provider/model/tier/tokens/cost_usd/outcome), and playbook.execute/tabular.execute with per-position/per-cell children. These are layered on top of the auto-instrumentation via a per-service observability_helpers.py (@traced + record_attributes) and are no-ops when OTel is disabled (the default). Span attributes carry counts and types only, never raw entity values — the anonymization promise is not allowed to regress through the telemetry side-channel (enforced by gateway/tests/test_anonymization_observability.py).
  • Database migrations and schema management — Alembic for the FastAPI backend, similar in the gateway. Operational detail, not architecture.
  • Container orchestration (Docker Compose for development, Helm for Kubernetes production) — deployment topology rather than runtime architecture. The Mermaid diagram shows logical services (Document Pipeline, Playbook Service, etc.); at runtime, long-running work for several of these services is hosted in dedicated arq worker processes that share Redis with the API container but consume from disjoint queues. M2 shipped ingest-worker (Document Pipeline) consuming the ingest queue; M3-A6 adds arq-worker consuming arq:m3a6, which hosts the Easy Playbook generation and Tabular Review jobs; M4 adds the Autonomous Layer’s autonomous_session_job plus two cron jobs (autonomous_idle_watchdog, autonomous_schedule_dispatcher) to that same arq-worker / arq:m3a6 queue. Queue isolation prevents a long playbook generation, tabular run, or autonomous session from blocking document ingest and vice versa. The two worker processes run the same api image with different WorkerSettings (app.workers.document_pipeline.WorkerSettings for ingest-worker; app.workers.arq_setup.WorkerSettings for arq-worker). Because autonomous sessions execute skills, the arq-worker now mounts the skills corpus (./skills:/skills:ro + LQ_AI_SKILLS_DIR) and installs the skill registry at startup via the shared app/skills/bootstrap.py::install_skill_registry — the same bootstrap the api lifespan uses, with uniform fail-fast on a missing/unreadable skills dir (post-v0.4.0, #139). SIGHUP-driven skill reload stays api-only: a running worker keeps its boot-time registry snapshot until it is restarted.
  • Citation Engine internals — within the Document Pipeline service, the Citation Engine is a multi-stage process (structured generation → deterministic substring verification → tolerant-match verification → paraphrase-judge verification → side-panel rendering). The chat surface renders each emitted citation in one of five visual states (M2-C2): verified-exact and verified-tolerant (both green), verified-paraphrase (yellow), unverified (greyed text + [unverified] marker), and system-error (yellow warning; deferred to M2-D when the pipeline emits the signal). The visual contract is load-bearing for procurement review — a reviewer scanning the report should be able to identify unverified citations without reading the tooltips. PRD §3.3 covers the engine internals in detail.


Diagram and document maintained alongside the canonical PRD. Updates land in the same release cadence as the PRD; non-trivial changes warrant a PRD version bump.