title: Configuration settings description: Every setting the backend and the Inference Gateway read, generated from their Pydantic settings classes, plus the two example configuration files in full. url: https://legalquants.github.io/lq-ai/reference/configuration/ checked-against: 671eab0d (2026-10-25) status: draft kind: explanation sources: docs/site/reference/configuration.intro.md, api/app/config.py, gateway/app/config.py, gateway.yaml.example, mcp.yaml.example Two things the tables below can't tell you on their own. **Tier numbers are security levels — lower is stronger.** Tier 1 (local, air-gap-capable) is the strongest posture; Tier 5 (consumer or free endpoints) is the weakest. Setting a `minimum_inference_tier` of N on a request, a project, or a skill means "Tier N or stronger": the gateway allows a routed tier of N or any lower number and refuses anything weaker with HTTP 403 (`tier_below_minimum`); when more than one declaration applies at once, the strictest (lowest number) wins. A tier is also only as good as what the operator says it is — the `tier` on a provider entry, or an `inference_tiers` override for a provider or a single model, is a claim about the hosting and contract behind that account, and nothing here checks it. The same provider can be Tier 4 under a standard commercial account and Tier 3 under a zero-data-retention agreement; which one actually applies is on the operator to verify. **A setting existing here doesn't mean a running container receives it.** The backend table lists everything `api/app/config.py` can read, but `docker-compose.yml` only forwards an explicit list of variables into each service — not the whole root `.env`. `LQ_AI_CHAT_HISTORY_TOKEN_BUDGET` and `LQ_AI_CHAT_HISTORY_MAX_MESSAGES`, for example, are both in `.env.example` and in the table below, yet the default Compose file doesn't pass either to `api` as of the checked commit — set them in `.env` alone and nothing changes. To actually use a setting that isn't forwarded, add it to the service's `environment:` block and recreate the service. ## Backend API — environment settings Parsed from [`api/app/config.py`](https://github.com/LegalQuants/lq-ai/blob/671eab0d/api/app/config.py) at the commit this page was checked against. ### `Settings` Backend API configuration. Field grouping mirrors `.env.example`. Only fields the backend reads are declared here — provider keys and Mode-2 / Ollama variables are read by the gateway, not by `api/`. **Postgres** | Setting | Type | Default | Description | |---|---|---|---| | `database_url` | `str` | `"postgresql+asyncpg://lq_ai:lq_ai@localhost:5432/lq_ai"` | Async SQLAlchemy URL for Postgres. | **Redis** | Setting | Type | Default | Description | |---|---|---|---| | `redis_url` | `str` | `"redis://localhost:6379/0"` | Redis URL used for sessions, queues, and rate limits. | **S3-compatible object storage** | Setting | Type | Default | Description | |---|---|---|---| | `s3_endpoint_url` | `str` | `"http://localhost:9000"` | S3-compatible endpoint URL (RustFS in Compose; operator-supplied S3 allowed). | | `s3_access_key` | `str` | `""` | S3 access key. | | `s3_secret_key` | `str` | `""` | S3 secret key. | | `s3_bucket` | `str` | `"lq-ai-files"` | S3 bucket for uploaded files. | | `s3_region` | `str` | `"us-east-1"` | S3 region. | **File upload limits (Task C4)** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_max_upload_size_mb` | `int` | `100` | Per-request cap on uploaded-file size in MB. M1 default: 100. Streamed enforcement; never loads the body into memory to measure. Operators raising this must also raise their ingress's body-size limit. | **Document pipeline (Task C5)** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_ingest_worker_concurrency` | `int` | `2` | Concurrency of the document-pipeline arq worker. Each job runs Docling + PyMuPDF (CPU-bound); 2 is conservative. | | `lq_ai_docling_timeout_seconds` | `int` | `300` | Per-job timeout for the document pipeline (Docling + PyMuPDF + chunking + persistence). Default 300 seconds. | | `lq_ai_docling_enabled` | `bool` | `True` | When True (default), run Docling for structured-content extraction. When False, skip Docling and use PyMuPDF only for offsets and content. | | `lq_ai_chunk_target_chars` | `int` | `2_000` | Target chunk size in characters. The chunker snaps the actual boundary to a sentence terminator within a 200-char lookback when possible. | | `lq_ai_chunk_overlap_chars` | `int` | `200` | Characters of overlap between consecutive chunks. Aids boundary-spanning citations during retrieval. | **Inference Gateway** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_gateway_url` | `str` | `"http://localhost:8001"` | Inference Gateway base URL. | | `lq_ai_gateway_key` | `str` | `""` | Shared secret for backend ↔ gateway. Required in prod. | | `lq_ai_gateway_timeout_seconds` | `float` | `900.0` | HTTP timeout in seconds for backend to Inference Gateway calls (ADR 0027). Must exceed the gateway's adapter timeouts (600 seconds by default) so the gateway can report failures first. Connection establishment is capped at 10 seconds. Restart API and workers after changing this deployment setting. (gt 0) | **Chat history (multi-turn memory)** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_chat_history_token_budget` | `int` | `64_000` | Approximate token budget (~4 chars/token) for prior chat turns replayed to the model. 0 disables multi-turn history. (ge 0) | | `lq_ai_chat_history_max_messages` | `int` | `20` | Hard cap on the number of prior chat messages replayed to the model, independent of the token budget. 0 disables history. (ge 0) | **Matter access control (migration 0067)** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_matter_default_share_scope` | `str` | `"personal"` | Ambient grant applied to a newly created matter when the caller does not set one. 'personal' (the default) is fail-restrictive: a new matter reaches only its owner until someone is added. 'org' gives every non-blocked user in the deployment READ on new matters, which suits a small firm where ambient visibility is the working assumption and an ethical screen is the exception. It never widens WRITE, and a 'blocked' membership row overrides it. Sandbox matters ignore this and stay 'personal' (DB CHECK). (pattern "^(personal\|members\|org)$") | **JWT (per ADR 0002 — backend owns auth)** | Setting | Type | Default | Description | |---|---|---|---| | `jwt_secret` | `str` | `DEV_JWT_SECRET` | Signing secret for JWT access and refresh tokens. | | `jwt_access_token_ttl_seconds` | `int` | `900` | Access-token TTL in seconds. Default: 15 minutes. | | `jwt_refresh_token_ttl_seconds` | `int` | `604800` | Refresh-token TTL in seconds. Default: 7 days. | | `session_absolute_timeout_seconds` | `int` | `28800` | Absolute session timeout in seconds. Copied verbatim across refresh-token rotations; the refresh endpoint 401s when exceeded. PRD §5.1 default: 8 hours. | | `session_idle_timeout_seconds` | `int` | `1800` | Idle session timeout in seconds. Refreshing the access token resets the clock; the refresh endpoint 401s when exceeded. PRD §5.1 default: 30 minutes. | **Autonomous (M4)** | Setting | Type | Default | Description | |---|---|---|---| | `autonomous_default_max_cost_usd` | `Decimal` | `Decimal("5.00")` | Global default per-session cost cap (USD) for autonomous sessions spawned by a watch or schedule that did not set max_cost_usd. Mirrors the gateway.yaml default. R4 (economic brake) trips when projected cost would exceed this cap. | | `autonomous_default_model` | `str` | `"smart"` | Fallback chat-completion model used by the autonomous analysis node when `params['model']` is not set on the session. Must be a model id or alias the gateway can route. | | `orchestration_chat_enabled` | `bool` | `False` | — | | `orchestration_chat_project_id` | `str` | `""` | — | | `orchestration_chat_provider` | `str` | `""` | — | | `orchestration_chat_model` | `str` | `""` | — | | `orchestration_chat_budget_usd` | `Decimal` | `Decimal("2.0000")` | — (gt 0) | | `orchestration_chat_minimum_tier` | `int` | `1` | — (ge 1, le 5) | | `orchestration_chat_timeout_seconds` | `int` | `120` | — (ge 1, le 900) | | `new_harness_enabled` | `bool` | `False` | ADR 0038 D1, D3 — New harness, the run profile the chat composer offers as an experimental model. It has its own switch and its own pinned route: enabling the fictional profile above never enables this one, and nothing is inherited from it. | | `new_harness_provider` | `str` | `""` | — | | `new_harness_model` | `str` | `""` | — | | `new_harness_budget_usd` | `Decimal` | `Decimal("5.0000")` | — (gt 0) | | `new_harness_minimum_tier` | `int` | `1` | — (ge 1, le 5) | | `new_harness_timeout_seconds` | `int` | `300` | — (ge 1, le 900) | | `new_harness_reasoning_effort` | `Literal["", "none", "minimal", "low", "medium", "high"]` | `""` | How much hidden reasoning a New harness call asks the model for. The gateway maps it to Ollama's think, sends it as it is to OpenAI and Azure OpenAI, and sends nothing for it to Anthropic. Hidden reasoning counts against the output limit. Empty, the default, sends nothing and leaves the model's own default. What a level does depends on the model: some treat every level as on, and a model that does not take the setting may refuse every call. Try a value against the pinned model before relying on it. | | `orchestration_deployment_children` | `int \| None` | `None` | Explicit shared child capacity. Required before enabling any run profile. (ge 1, le 32) | | `mfa_mandatory` | `bool` | `False` | Require MFA enrollment for every user. When True, non-enrolled users are gated to the MFA-setup endpoints until they enroll. | **Password hashing (per ADR 0002)** | Setting | Type | Default | Description | |---|---|---|---| | `bcrypt_rounds` | `int` | `12` | Bcrypt cost factor for password hashing. Default 12. | **First-run admin (per Task B2)** | Setting | Type | Default | Description | |---|---|---|---| | `first_run_admin_email` | `str` | `"admin@lq.ai"` | Email for the auto-created first-run admin user. Set before first deployment; ignored on subsequent restarts. | | `password_min_length` | `int` | `12` | Minimum length for user-set passwords. Default: 12 characters. | **MFA challenge token (per ADR 0002 / PRD §5.1)** | Setting | Type | Default | Description | |---|---|---|---| | `mfa_token_ttl_seconds` | `int` | `300` | MFA challenge token TTL in seconds. Default: 5 minutes. | **GDPR Article 17 grace period (per Task D6 / PRD §5.3)** | Setting | Type | Default | Description | |---|---|---|---| | `gdpr_grace_period_days` | `int` | `30` | Days between a user's account-deletion request and hard deletion. 0 hard-deletes on the next worker tick; the GDPR-typical default is 30. (ge 0) | **Skill registry (per Task C1 / ADR 0004)** | Setting | Type | Default | Description | |---|---|---|---| | `skills_dir` | `str` | `"../skills"` | Filesystem directory the skill loader walks at startup and on SIGHUP. Default is the repo's `skills/` folder. | | `community_skills_dir` | `str \| None` | `None` | Override for the community skills directory. Defaults to `skills/community/skills/` relative to the repo root. Set to an empty string to disable community skill loading. | **M3-D1 slack-bridge integration** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_bridge_token` | `str` | `""` | Shared bearer token the slack-bridge presents on POSTs to /api/v1/integrations/slack/workspaces. Constant-time matched. | | `lq_ai_bridge_master_key` | `str` | `""` | urlsafe-base64 Fernet master key used to encrypt Slack bot tokens at rest (and any future bridge-issued secret). | **Chat tool-loop (PR5b / L4)** | Setting | Type | Default | Description | |---|---|---|---| | `chat_tool_call_cap` | `int` | `8` | Maximum number of tool calls the chat tool-loop will execute in a single turn before issuing a final no-tools round (PRD §L4). Default: 8. Operator-overridable via LQ_AI_CHAT_TOOL_CALL_CAP. (ge 1) | **Tool-egress ceiling (issue #593)** | Setting | Type | Default | Description | |---|---|---|---| | `tool_max_egress_tier` | `int \| None` | `None` | Operator default ceiling for tool egress tiers (1-5). Unset disables the operator ceiling. Operator-overridable via LQ_AI_TOOL_MAX_EGRESS_TIER. (ge 1, le 5) | | `skill_reference_read_enabled` | `bool` | `True` | Allow activated skills to read installed references/. Human inspection remains available. | | `skill_workspaces_enabled` | `bool` | `False` | — | | `skill_script_runner_url` | `str \| None` | `None` | — | | `skill_script_runner_token` | `str` | `""` | — | **Operational** | Setting | Type | Default | Description | |---|---|---|---| | `log_level` | `LogLevel` | `"info"` | Log level for the api/ service. | | `lq_ai_dev_mode` | `bool` | `False` | When true, relax some safety checks for local development. | **SMTP / email transport (M4-C1)** | Setting | Type | Default | Description | |---|---|---|---| | `smtp_host` | `str \| None` | `None` | SMTP server hostname for autonomous-notification email. Unset disables email transport (in-app notifications still work). | | `smtp_port` | `int` | `587` | SMTP server port. Default 587 (STARTTLS submission). | | `smtp_username` | `str \| None` | `None` | SMTP auth username. Unset skips login (open relay / no auth). | | `smtp_password` | `str \| None` | `None` | SMTP auth password. Unset skips login. | | `smtp_from` | `str \| None` | `None` | From address for notification email. Falls back to `smtp_username` when unset. | | `smtp_use_tls` | `bool` | `True` | Issue STARTTLS after connecting. Default True. | | `smtp_timeout` | `int` | `10` | Socket timeout (seconds) for the SMTP connection — applies to connect, STARTTLS, and send. Bounds the best-effort send so a hung/black-holing mail server can't tie up a worker thread. | **CORS** | Setting | Type | Default | Description | |---|---|---|---| | `lq_ai_cors_origins` | `str` | `""` | Comma-separated allowed origins for CORS. Empty disables CORS. For local Compose dev set to http://localhost:3000. | ## Inference Gateway — `gateway.yaml` schema Parsed from [`gateway/app/config.py`](https://github.com/LegalQuants/lq-ai/blob/671eab0d/gateway/app/config.py) at the commit this page was checked against. ### `ServerConfig` Top-level `server:` block. **Server / auth** | Setting | Type | Default | Description | |---|---|---|---| | `host` | `str` | `"0.0.0.0"` | — | | `port` | `int` | `8001` | — (ge 1, le 65535) | | `log_level` | `LogLevel` | `"info"` | — | ### `GatewayAuthConfig` `gateway_auth:` — how the backend authenticates to the gateway. **Server / auth** | Setting | Type | Default | Description | |---|---|---|---| | `enabled` | `bool` | `True` | — | | `api_key_env` | `str` | `"LQ_AI_GATEWAY_KEY"` | — | ### `ProviderConfig` One entry under `providers:`. Keeps unknown provider-specific fields (`project_id`, `region`, `aws_region`, `api_version`, etc.) by allowing extra fields. The per-type adapter (B3+) is responsible for reading what it needs. API-key sourcing (per ADR 0011) supports two paths and exactly one is used per provider: * `api_key_env: ANTHROPIC_API_KEY` — read the key from the environment at adapter build time. Existing path; deprecated but still loadable through M1+M2. * `api_key_encrypted: gAAAAAB...` — Fernet-wrapped ciphertext; decrypted in-memory by `app.secrets.ProviderKeyResolver` using the master key bound at gateway startup (:envvar:`LQ_AI_GATEWAY_MASTER_KEY`). Mixing both on a single entry is rejected by the validator. **Providers** | Setting | Type | Default | Description | |---|---|---|---| | `name` | `str` | **required** | — (min_length 1) | | `type` | `ProviderType` | **required** | — | | `base_url` | `str` | **required** | — (min_length 1) | | `api_key_env` | `str \| None` | `None` | Some local providers (e.g., Ollama) legitimately have an empty string here. Accept `None` and `""` to mean "no key required". | | `api_key_encrypted` | `str \| None` | `None` | Fernet token (urlsafe-base64) of the provider key. Decrypted at gateway startup. See `app.secrets` and ADR 0011. | | `tier` | `InferenceTier` | **required** | — | | `models` | `list[str]` | `list()` | — | | `enabled` | `bool` | `True` | — | | `bedrock_protocol` | `BedrockProtocol \| None` | `None` | Explicit compatible API; unset preserves the reserved, unimplemented SigV4 slot. | | `bedrock_endpoint` | `BedrockEndpoint` | `"runtime"` | — | | `use_max_completion_tokens` | `bool` | `False` | Send `max_completion_tokens` instead of `max_tokens` to the provider. OpenAI/Azure GPT-5 and o-series reasoning deployments REJECT `max_tokens` (hard HTTP 400) and require `max_completion_tokens`; it is also accepted by gpt-4o. Default off so OpenAI-compatible servers that only recognize `max_tokens` (e.g. vLLM < 0.6.4 rejects unknown fields; LM Studio / Ollama / TGI silently drop them) keep working. Honored only by the OpenAI-family adapters (`openai` / `openai_compatible` / `azure_openai`). NB: `max_completion_tokens` caps reasoning + visible output combined — pair this with a generous budget on reasoning models or the response can come back empty with `finish_reason='length'`. | ### `EgressAllowlistConfig` Per-provider outbound host allowlist (the SSRF guard's allow set). **Tool / data-source providers (ADR 0014)** | Setting | Type | Default | Description | |---|---|---|---| | `hosts` | `list[str]` | **required** | Non-empty list of exact hostnames the provider may egress to. An empty allowlist is a misconfiguration, not 'allow all' — reject at config load. (min_length 1) | ### `ToolProviderRateLimitConfig` Per-provider rate limit, enforced at the adapter (ADR 0014). NOT the gateway's global `rate_limits` (whose enforcement middleware is unwired). This is a self-contained per-provider limit applied by `Router.route_tool_call` before each outbound call. **Tool / data-source providers (ADR 0014)** | Setting | Type | Default | Description | |---|---|---|---| | `requests_per_minute` | `int` | `60` | — (ge 1) | ### `ToolProviderConfig` One entry under `tool_providers:` (ADR 0014 D1). Sibling of `ProviderConfig`, not a subclass — a tool provider is invoked via `invoke_tool`, not `chat_completion`. Reuses the same two API-key sourcing paths as inference providers (ADR 0011). **Tool / data-source providers (ADR 0014)** | Setting | Type | Default | Description | |---|---|---|---| | `name` | `str` | **required** | — (min_length 1) | | `type` | `ToolProviderType` | **required** | — | | `base_url` | `str` | **required** | — (min_length 1) | | `api_key_env` | `str \| None` | `None` | — | | `api_key_encrypted` | `str \| None` | `None` | — | | `egress_tier` | `InferenceTier` | **required** | Data-egress tier (ADR 0014 D4). The gateway refuses a call whose matter/skill ceiling is more restrictive (a lower tier number) than this provider's egress_tier. | | `allowlist` | `EgressAllowlistConfig` | **required** | — | | `rate_limit` | `ToolProviderRateLimitConfig` | `ToolProviderRateLimitConfig()` | — | | `anonymize_outbound` | `bool` | `True` | Default True per ADR 0014 D5. Required for checked authority-query anonymization. Legacy calls do not yet apply this transform automatically. | | `enabled` | `bool` | `True` | — | | `user_agent` | `str \| None` | `None` | Descriptive `User-Agent` header value, required for `type: edgar` (SEC's fair-access policy substitutes for an API key) and `type: eurlex` (the EU Publications Office Cellar service has no API-key scheme at all). Unused by other provider types. | ### `MCPServerConfig` One entry under `mcp_servers:` in `mcp.yaml`. Synthesized into a `type: mcp` `ToolProviderConfig` at load (WS2 spec D2). **MCP server config (PR4a)** | Setting | Type | Default | Description | |---|---|---|---| | `name` | `str` | **required** | — (min_length 1) | | `server_url` | `str` | **required** | — (min_length 1) | | `auth` | `MCPAuthType` | `"none"` | — | | `api_key_env` | `str \| None` | `None` | — | | `api_key_encrypted` | `str \| None` | `None` | — | | `oauth_client_id` | `str \| None` | `None` | Pre-registered public OAuth client_id for this MCP server. Required when `auth == "oauth"`; ignored otherwise. The api reads this via the sanitized `GET /admin/v1/config` response to initiate the out-of-band authorization-code flow on behalf of the user (PR4c). | | `egress_tier` | `InferenceTier` | **required** | — | | `allowlist` | `EgressAllowlistConfig` | **required** | — | | `rate_limit` | `ToolProviderRateLimitConfig` | `ToolProviderRateLimitConfig()` | — | | `enabled` | `bool` | `True` | — | ### `ModelTarget` A specific (provider, model) target referenced from an alias. **Model aliases** | Setting | Type | Default | Description | |---|---|---|---| | `provider` | `str` | **required** | — (min_length 1) | | `model` | `str` | **required** | — (min_length 1) | ### `ModelAliasConfig` One entry under `model_aliases:`. Per PRD §4.4 / `gateway.yaml.example`, an alias has a `primary` target and an optional `fallback` chain. Skills and the backend reference aliases (`smart`, `fast`, `budget`, `local`, `embedding`) rather than concrete provider/model pairs. **Model aliases** | Setting | Type | Default | Description | |---|---|---|---| | `primary` | `ModelTarget` | **required** | — | | `fallback` | `list[ModelTarget]` | `list()` | — | ### `InferenceTiersConfig` `inference_tiers:` — operator-controlled tier-derivation overrides. Two layers of resolution applied by :func:`derive_routed_inference_tier`: 1. `overrides` — exact `"/"` keys win. Used when a single (provider, model) pair lands at a non-default tier (e.g., a ZDR'd Claude account on Bedrock that should be Tier 2 even though Bedrock's default is 3). 2. `defaults` — per `ProviderType` (`anthropic`, `openai`, `vertex`, ...). Used when an operator wants every model under a given provider type to land at a non-default tier. Both layers are *optional*. If neither matches, the tier falls back to the provider entry's own `tier:` field (the simple posture documented in `gateway.yaml.example`). Per PRD §4.4 / §1.5.2 / §3.13. Tier values are 1-5 inclusive. **Inference-tier derivation (B4)** | Setting | Type | Default | Description | |---|---|---|---| | `defaults` | `dict[ProviderType, InferenceTier]` | `dict()` | Per-provider-type tier defaults (`anthropic` → 4, `openai` → 4, `ollama` → 1, ...). Keys are validated against the `ProviderType` enum so a typo is caught at startup, not on the first inference call. | | `overrides` | `dict[str, InferenceTier]` | `dict()` | Per-(provider, model) tier overrides keyed as `"/"`. Example:: overrides: anthropic-prod/claude-opus-4-7: 3 # operator's ZDR addendum A bare `""` key is also honored; it lifts every model on that named provider to the override value (rare; per-type `defaults` is usually the cleaner expression of operator intent). | ### `TierPolicyConfig` `tier_policy:` — operator-level tier policy. A3 loads this; D1 enforces `default_minimum_tier` / `privileged_minimum_tier` refusals. **Tier policy** | Setting | Type | Default | Description | |---|---|---|---| | `allowed_tiers_global` | `list[InferenceTier]` | `lambda: [1, 2, 3, 4]()` | — | | `default_minimum_tier` | `InferenceTier` | `4` | — | | `privileged_minimum_tier` | `InferenceTier` | `3` | — | | `warn_on_tiers` | `list[InferenceTier]` | `lambda: [4, 5]()` | — | ### `AnonymizationConfig` `anonymization:` block. M2 feature. Typed fields the M2-B3 middleware reads directly: * `enabled` — master switch. `False` short-circuits the pre/post middleware to a no-op regardless of per-request `anonymize` flags or tier gating. * `apply_at_tiers` — tier gate. The middleware fires only when the request's routed tier is in this list. Tiers outside `[1, 5]` raise a config-load error rather than silently becoming a no-op. Other keys from `gateway.yaml.example` (`entity_types` etc.) pass through under `extra="allow"` and are documented in `docs/security/anonymization.md`; M2-B3 does not depend on them yet. **Rate limits / cost / validation / etc. (loose schemas)** | Setting | Type | Default | Description | |---|---|---|---| | `enabled` | `bool` | `False` | — | | `apply_at_tiers` | `list[int]` | `list()` | — | ### `CostRateEntry` One `cost_tracking.rates` entry: input/output USD per million tokens. The B4 router uses these to populate `inference_routing_log.cost_estimate` for each routed request. If the resolved `"/"` key is not in `CostTrackingConfig.rates`, `cost_estimate` is left `NULL` rather than invented (per CLAUDE.md: don't overclaim). | Setting | Type | Default | Description | |---|---|---|---| | `input_per_mtok` | `float` | **required** | — (ge 0.0) | | `output_per_mtok` | `float` | **required** | — (ge 0.0) | ### `CostTrackingConfig` | Setting | Type | Default | Description | |---|---|---|---| | `enabled` | `bool` | `True` | — | | `rates` | `dict[str, CostRateEntry]` | `dict()` | Per-(provider, model) USD-per-million-token rates keyed as `"/"`. See `gateway.yaml.example` for the canonical shape. The router consults this when computing `cost_estimate` for the `inference_routing_log` row. | ### `CircuitBreakerConfig` | Setting | Type | Default | Description | |---|---|---|---| | `enabled` | `bool` | `True` | — | ### `RequestValidationConfig` | Setting | Type | Default | Description | |---|---|---|---| | `max_max_tokens` | `int` | `16384` | — (ge 1) | | `max_messages_per_request` | `int` | `100` | — (ge 1) | | `max_total_request_chars` | `int` | `1_000_000` | — (ge 1) | | `large_context_threshold_tokens` | `int` | `50_000` | — (ge 1) | ### `DevModeConfig` | Setting | Type | Default | Description | |---|---|---|---| | `enabled` | `bool` | `False` | — | ### `EnsembleVerificationConfig` `citation_engine.ensemble_verification:` block — M2-D1. Stage 4 of the Citation Engine cascade. When activated (per-skill, per-project, or per-deployment), the paraphrase judge runs in parallel across `judge_models` and the verdicts aggregate under `aggregation_rule`. Activation is computed by the api/ at chat-send time: `any()` across the skill frontmatter `ensemble_verification` flag, the project's `ensemble_verification` column, and this block's `default_enabled`. The gateway exposes the resolved config via `GET /v1/citation-engine/config`; the api/ caches it for the process and reads it on every verify() call. `judge_models` is a list of aliases (or fully-qualified `provider/model`) the gateway's router can resolve. Three is the canonical M2-D1 baseline (primary judge + 2 alternates from different provider families); operators may choose more or fewer. An empty list disables ensemble even when `default_enabled` is True — defense-in-depth against misconfiguration. `aggregation_rule` selects strict (all judges must verdict `"yes"`) or majority (simple majority wins). The persisted `message_citations.verification_method` is `'ensemble_strict'` or `'ensemble_majority'` accordingly. `max_cost_per_message_usd` is the pre-flight budget cap. The api/ estimates `input_tokens * len(judge_models) * max_judge_rate` before dispatch; if the estimate exceeds the cap, verification falls back to a single Stage 3 judge call with a warning logged. Default `0.05` is a conservative ceiling; operators tune per their cost posture. | Setting | Type | Default | Description | |---|---|---|---| | `default_enabled` | `bool` | `False` | — | | `judge_models` | `list[str]` | `list()` | — | | `aggregation_rule` | `Literal["strict", "majority"]` | `"strict"` | — | | `max_cost_per_message_usd` | `float` | `0.05` | — (ge 0.0) | ### `CitationEngineConfig` `citation_engine:` block — M2-C1 / M2-D1. The Citation Engine runs in the api/ but reads the *model name* it should use for the Stage 3 (paraphrase) judge from the gateway's config so the operator has one place to configure model choices. The api/ pulls this over `GET /v1/citation-engine/config` at startup (or on first need) and caches it for the process. `judge_model` accepts any value the gateway's router can resolve: an alias from `model_aliases` (`fast`, `smart`, `budget`) or a fully-qualified provider/model form (`anthropic-prod/claude-haiku-4-5`). Defaults to `"fast"` — deliberately a smaller model than the typical citation-generating model so the judge cost is bounded. `ensemble_verification` (M2-D1) is the Stage 4 ensemble block. See `EnsembleVerificationConfig` for the per-field contract. Default-constructed (`default_enabled=False`, empty `judge_models`) means Stage 4 is opt-in per skill or project. | Setting | Type | Default | Description | |---|---|---|---| | `judge_model` | `str` | `"fast"` | — (min_length 1) | | `ensemble_verification` | `EnsembleVerificationConfig` | `EnsembleVerificationConfig()` | — | ### `GatewayConfig` Full `gateway.yaml` shape. Loaded once on startup by :func:`app.config_loader.load_config` and attached to `app.state.config`. Treat as immutable after startup. **Top-level config** | Setting | Type | Default | Description | |---|---|---|---| | `server` | `ServerConfig` | `ServerConfig()` | — | | `gateway_auth` | `GatewayAuthConfig` | `GatewayAuthConfig()` | — | | `providers` | `list[ProviderConfig]` | `list()` | — | | `tool_providers` | `list[ToolProviderConfig]` | `list()` | — | | `model_aliases` | `dict[str, ModelAliasConfig]` | `dict()` | — | | `inference_tiers` | `InferenceTiersConfig` | `InferenceTiersConfig()` | — | | `tier_policy` | `TierPolicyConfig` | `TierPolicyConfig()` | — | | `rate_limits` | `RateLimitsConfig` | `RateLimitsConfig()` | — | | `anonymization` | `AnonymizationConfig` | `AnonymizationConfig()` | — | | `citation_engine` | `CitationEngineConfig` | `CitationEngineConfig()` | — | | `cost_tracking` | `CostTrackingConfig` | `CostTrackingConfig()` | — | | `telemetry` | `TelemetryConfig` | `TelemetryConfig()` | — | | `circuit_breaker` | `CircuitBreakerConfig` | `CircuitBreakerConfig()` | — | | `request_validation` | `RequestValidationConfig` | `RequestValidationConfig()` | — | | `dev_mode` | `DevModeConfig` | `DevModeConfig()` | — | ## `gateway.yaml.example` Reproduced in full from [`gateway.yaml.example`](https://github.com/LegalQuants/lq-ai/blob/671eab0d/gateway.yaml.example), comments included — the comments are the operator documentation. ```yaml # ============================================================ # LQ.AI — Inference Gateway Configuration # ============================================================ # This file configures the Inference Gateway: which providers # are available, which models map to which provider, what tier # each routing decision lands at, anonymization, rate limits, # and operational settings. # # Save your operator-specific copy as `gateway.yaml` (this # `.example` file is committed to the repo as reference). # # References: # PRD §4 — Inference Gateway specification # PRD §1.5.2 — Inference Choice Spectrum (the 5-tier model) # PRD §4.7 — Anonymization Layer (M2) # # Configuration changes take effect on gateway restart for # top-level changes; tier-policy changes via /admin/inference-tiers # take effect immediately and are audited. # ============================================================ # Gateway operational settings server: host: 0.0.0.0 port: 8001 log_level: info # debug | info | warn | error # How the gateway authenticates incoming requests # (the backend service identifies itself with this key) gateway_auth: enabled: true api_key_env: LQ_AI_GATEWAY_KEY # Backend services must include X-LQ-AI-Gateway-Key header # # The gateway uses the SAME shared secret in the reverse direction # when fetching skill content from the backend's # `/api/v1/internal/skills/{name}` endpoint during prompt assembly # (C2 / ADR 0006). The api/ side also reads LQ_AI_GATEWAY_KEY and # constant-time compares the X-LQ-AI-Gateway-Key header on inbound # internal-route calls. # # Skill-fetch base URL: LQ_AI_API_URL (env var; defaults to # http://api:8000 in the Compose stack). Skill-fetch cache TTL: # LQ_AI_SKILL_CACHE_TTL_SECONDS (env var; default 60s). # ============================================================ # PROVIDERS — what LLM endpoints the gateway can reach # ============================================================ # Each provider entry includes: # name — operator-chosen identifier (referenced from model_aliases) # type — provider family (anthropic, openai, vertex, etc.) # base_url — endpoint URL # api_key_env — environment variable name to look up (NOT the key) # api_key_encrypted — Fernet-wrapped ciphertext (ADR 0011, alternative) # tier — default Inference Tier for this provider/account # models — which models the provider exposes # # Per ADR 0011 each provider entry uses EITHER `api_key_env` OR # `api_key_encrypted`, not both. The encrypted path lets operators # commit `gateway.yaml` to a private repo / backup snapshots without # leaking the plaintext key: # 1. `python -m app.cli generate-master-key` → store as # LQ_AI_GATEWAY_MASTER_KEY in your secrets vault. # 2. Export the master key in the gateway's environment. # 3. `python -m app.cli encrypt-key --provider ` → paste the # output as `api_key_encrypted: gAAAAAB...`. # The gateway decrypts in-memory at adapter build time; plaintext keys # never touch disk after step 3. # # THIRD PATH — runtime admin provider-keys API (post-v0.4.0, #128). # Admins can manage keys at runtime via `/admin/v1/provider-keys` # (proxied by the backend at `/api/v1/admin/provider-keys`) instead of # hand-editing this file: POST writes the `api_key_encrypted` field into # this `gateway.yaml` automatically (same Fernet wrapping as step 3) AND # hot-applies the key to the live adapter in-place — no restart, no # SIGHUP. PATCH rotates, DELETE revokes. This path requires # LQ_AI_GATEWAY_MASTER_KEY to be set (the API returns 400 # failed_precondition otherwise). Keys provided via `api_key_env` # continue to work and report their source as `env`; runtime-managed # keys report `runtime`. The encrypt-CLI path above remains valid for # operators who prefer to commit ciphertext by hand. # # IMPORTANT: tier values reflect the operator's specific # agreement with the provider. A standard commercial Anthropic # API account is Tier 4. An Anthropic enterprise account with # zero-data-retention is Tier 3. Local Ollama is Tier 1. The # operator MUST verify their tier assignments match their # actual provider relationships. providers: # --- Cloud providers --- - name: anthropic-prod type: anthropic base_url: https://api.anthropic.com api_key_env: ANTHROPIC_API_KEY tier: 4 # Standard commercial Anthropic API # tier: 3 # If operator has enterprise ZDR agreement # Optional: default max_tokens injected when the caller omits it # (Anthropic requires the field). Falls back to 16384 when unset — a # ceiling, not a spend; an explicit request max_tokens always wins. # Operator escape hatch (ADR 0027 D2), not the tuning axis. # default_max_tokens: 8192 models: - claude-opus-4-7 - claude-sonnet-4-6 - claude-haiku-4-5 - name: openai-prod type: openai base_url: https://api.openai.com/v1 api_key_env: OPENAI_API_KEY tier: 4 # Standard commercial OpenAI API # tier: 3 # If operator has zero-data-retention contract # GPT-5 / o-series reasoning models REJECT max_tokens (hard 400) and # require max_completion_tokens. Leave false for gpt-4o / older models # or OpenAI-compatible local servers that only accept max_tokens. use_max_completion_tokens: true models: - gpt-5 - gpt-5-mini - gpt-5-nano - name: vertex-anthropic type: vertex base_url: https://us-central1-aiplatform.googleapis.com api_key_env: GOOGLE_APPLICATION_CREDENTIALS # path to service-account JSON # `:-` default lets the gateway load even when GCP_PROJECT_ID is unset # (fresh `docker compose up` works without GCP credentials). Operators # routing to Vertex set GCP_PROJECT_ID in their .env. project_id: ${GCP_PROJECT_ID:-disabled} region: us-central1 tier: 3 # Vertex routes through customer's GCP project; ZDR by GCP terms models: - claude-opus-4-7@anthropic - claude-sonnet-4-6@anthropic - name: bedrock type: bedrock # One protocol per entry: messages, chat_completions, or responses. # Additional entries may share this key with different explicit protocols. # No SigV4/STS credentials are consumed by the compatible-API adapter. bedrock_protocol: messages bedrock_endpoint: runtime # mantle is an explicit opt-in for models requiring it base_url: https://bedrock-runtime.${AWS_REGION:-us-east-1}.amazonaws.com/anthropic api_key_env: AWS_BEARER_TOKEN_BEDROCK tier: 3 # Operator must verify terms, retention/logging and regional requirements. models: # Select an entitled model/profile for the configured region; this is not # a claim that the account has access. See docs/bedrock-verification.md. - us.anthropic.claude-haiku-4-5-20251001-v1:0 - name: azure-openai type: azure_openai # M2-E1 (DE-267): Azure OpenAI adapter. Mirrors the OpenAI wire shape # with a deployment-scoped URL and `api-key` auth. # # `:-` default lets the gateway load without an Azure resource configured; # operators routing to Azure set AZURE_OPENAI_RESOURCE in their .env. base_url: https://${AZURE_OPENAI_RESOURCE:-disabled}.openai.azure.com api_key_env: AZURE_OPENAI_API_KEY # api_version pins which Azure-side features are available (Azure rolls # capabilities in/out per version). Bump to a newer GA version # ('2024-10-21' or later) for current chat-completion features. api_version: '2024-08-01-preview' tier: 3 # Azure OpenAI under enterprise agreement (ZDR + BAA) # GPT-5 / o-series deployments REJECT max_tokens (hard 400) and require # max_completion_tokens. Azure deployment-ids are operator-named, so this # is set per-provider rather than inferred from the model name. use_max_completion_tokens: true # CRITICAL: each entry is the Azure DEPLOYMENT-ID (operator-named in # the Azure portal), NOT the underlying model name. Many operators # run multiple deployments of the same model with different SKUs; # the gateway routes by deployment-id via the alias map below. models: - gpt-5-prod # e.g., a deployment of gpt-5 - gpt-5-mini-prod # e.g., a deployment of gpt-5-mini # --- Local providers (Tier 1) --- - name: ollama-local type: ollama # Default reaches the in-Compose ollama service (pulled in by # `docker compose --profile local up`). Operators running Ollama # on the host instead set OLLAMA_BASE_URL=http://host.docker.internal:11434 # in their .env. Per PRD §1.5.1 / §6.1 Ollama is the Mode-2 (fully # local, air-gap-capable) backbone — Tier 1 by every authoritative # reference (PRD §1.5.2: "Local-only inference (air-gap-capable)"). # See B6 partial in M1-PROGRESS for the adapter that wires this up. # # NOTE (LLM egress policy, GW-04): a plaintext ``http://`` base_url is # accepted only for loopback/private/local hosts (localhost, RFC-1918 # ranges, host.docker.internal, in-Compose service names). A base_url # that violates the policy is FATAL AT STARTUP — the gateway refuses to # boot rather than silently skip the provider, because a skipped local # primary would let the router fall through to a cloud fallback the # operator did not choose. If startup fails naming this provider, fix # the base_url (most often a mistyped OLLAMA_BASE_URL). base_url: ${OLLAMA_BASE_URL:-http://ollama:11434} api_key_env: '' # Ollama doesn't require an API key tier: 1 # Fully local; no data leaves the deployment models: - qwen3.5:9b - qwen3.5:4b-nvfp4 - gemma4:e4b enabled: ${LOCAL_INFERENCE_ENABLED:-true} - name: vllm-local type: openai_compatible # vLLM exposes OpenAI-compatible API base_url: http://vllm:8000/v1 # Empty: a keyless local server. For a hosted OpenAI-compatible service, # name its key variable here; if that variable is not set the provider # is skipped at startup with a warning, like any other missing key. api_key_env: '' tier: 1 models: - meta-llama/Meta-Llama-3.1-70B-Instruct enabled: ${VLLM_ENABLED:-false} # ============================================================ # TOOL / DATA-SOURCE PROVIDERS (ADR 0014) — third-party egress # ============================================================ # A tool provider is third-party egress the gateway brokers and audits # (case-law APIs, MCP servers). Distinct from inference `providers:` above. # Every entry is SSRF/allowlist-guarded, tier-tagged, rate-limited, and # written to `tool_egress_log`. Uncomment and configure to enable. # # Runtime alternative: enable these from the app (Settings → Research sources) # or POST /api/v1/admin/tool-providers — no YAML edit or restart needed. # # tool_providers: # - name: courtlistener-prod # type: courtlistener # courtlistener type is shipped (WS3a); uncomment + set COURTLISTENER_API_TOKEN to enable # base_url: https://www.courtlistener.com/api/rest/v4 # api_key_env: COURTLISTENER_API_TOKEN # OR api_key_encrypted (ADR 0011) # egress_tier: 4 # data-egress tier (ADR 0014 D4) # allowlist: # hosts: [www.courtlistener.com] # exact outbound host allowlist (SSRF) # rate_limit: # requests_per_minute: 60 # per-provider, enforced at the adapter # anonymize_outbound: true # default; generic CourtListener transform remains deferred # # - name: govinfo-prod # type: govinfo # shipped in WS-E PR1a; uncomment + set GOVINFO_API_KEY to enable # base_url: https://api.govinfo.gov # api_key_env: GOVINFO_API_KEY # api.data.gov key — free tier sufficient for legal research # egress_tier: 4 # public statutory/regulatory data (ADR 0014 D4) # allowlist: # hosts: [api.govinfo.gov] # exact outbound host allowlist (SSRF) # rate_limit: # requests_per_minute: 60 # api.data.gov default is 1200/min; 60 is conservative # anonymize_outbound: false # opt-out; set true for checked authority-query anonymization # # GovInfo data is free (no per-call cost). Fields below are optional DE-344 extensions # # for future cost-tracking dashboards; omit or set to 0 for free sources: # # cost_per_call: 0.0 # # cost_per_unit: 0.0 # # - name: edgar-prod # type: edgar # shipped in WS-E PR2a; uncomment to enable # base_url: https://efts.sec.gov # full-text search host; get_authority reaches www.sec.gov # user_agent: "YourOrg legal-ops@yourorg.example" # SEC fair-access policy — REQUIRED, no API key # egress_tier: 4 # public filing data (ADR 0014 D4) # allowlist: # hosts: [efts.sec.gov, www.sec.gov] # rate_limit: # requests_per_minute: 300 # SEC allows ~10 req/s # anonymize_outbound: false # opt-out; set true for checked authority-query anonymization # # (no api_key_env — EDGAR needs only a User-Agent) # # (no cost_per_call — free source; R4 stays a no-op) # # - name: eurlex-prod # type: eurlex # shipped in WS-E PR2b; uncomment to enable # base_url: https://publications.europa.eu # EU Cellar (content-negotiation by CELEX) # user_agent: "YourOrg legal-ops@yourorg.example" # descriptive UA — REQUIRED, no API key # egress_tier: 4 # public EU legal text (ADR 0014 D4) # allowlist: # hosts: [publications.europa.eu] # rate_limit: # requests_per_minute: 60 # anonymize_outbound: false # opt-out; set true for checked authority-query anonymization # # (no api_key_env — EUR-Lex Cellar needs only a User-Agent) # # (no cost_per_call — free source; R4 stays a no-op) # ============================================================ # MODEL ALIASES — friendly names → specific provider/model # ============================================================ # Skills and the backend reference models by alias rather than # by specific provider/model. This decouples skill authoring # from provider configuration; the operator can swap out the # provider behind 'smart' without changing any skill. # # RAW PASSTHROUGH (D0): callers may also bypass the alias map by # sending ``model: "/"`` directly, # e.g., ``anthropic-prod/claude-haiku-4-5`` or # ``ollama-local/qwen2.5:7b``. The provider name MUST match an # entry in ``providers:`` above; the model component is the # provider-native name (no further translation). Tier is derived # the same way as for aliases — overrides → defaults → # provider.tier. Used by the LQ.AI shell's model picker so users # can target a specific model without authoring a new alias. model_aliases: # User-facing aliases referenced by skills and the chat UI smart: primary: provider: anthropic-prod model: claude-opus-4-7 fallback: - {provider: vertex-anthropic, model: claude-opus-4-7@anthropic} - {provider: openai-prod, model: gpt-5} fast: primary: provider: anthropic-prod model: claude-sonnet-4-6 fallback: - {provider: openai-prod, model: gpt-5-mini} budget: primary: provider: anthropic-prod model: claude-haiku-4-5 fallback: - {provider: openai-prod, model: gpt-5-nano} local: primary: provider: ollama-local model: qwen3.5:9b fallback: [] # No fallback for local — Tier 1 must stay Tier 1 # Mode-2 friendly aliases (B6 partial). Skills and the chat UI ask # for these by name when the operator wants the response to come from # the air-gapped local model. Both stay Tier 1 — the fallback list is # empty so a missing local provider surfaces as a clean 503 rather # than silently degrading to a cloud Tier 4 path. # # Default model choices reflect what a current Ollama install can # actually serve well in 2026. Qwen3.5 is the family we ship behind # the Mode-2 aliases (PRD §6.1); operators free to repoint at any # ollama-served model by editing this section. The "fast" / "thinking" # split mirrors the cloud split — 4b for low-latency, 9b when the # quality lift is worth the wall-clock. local-fast: primary: provider: ollama-local model: qwen3.5:4b-nvfp4 fallback: [] local-thinking: primary: provider: ollama-local model: qwen3.5:9b fallback: [] # The 'embedding' alias is the alias the backend's KB layer (C6) # asks the gateway to dispatch. Per ADR 0008 the default points at # OpenAI's text-embedding-3-small (1536-dim — matches the # document_chunks.embedding column type). Operators who don't run # OpenAI can repoint at a different provider when its adapter # lands (e.g., a future Voyage adapter) without changing any # application code; the dimension constraint must match the column # type or the operator runs an ALTER migration to swap the column. embedding: primary: provider: openai-prod model: text-embedding-3-small fallback: [] # ============================================================ # INFERENCE TIERS — derivation overrides (PRD §1.5.2 / §3.13) # ============================================================ # Tier derivation runs for every routed request and stamps # ``routed_inference_tier`` on the response (header + body # field) and into ``inference_routing_log``. Resolution order # (first match wins): # # 1. inference_tiers.overrides["/"] # 2. inference_tiers.overrides[""] # 3. inference_tiers.defaults[""] # 4. The provider entry's own ``tier:`` field (above) # # This block is OPTIONAL. With the block omitted, every routed # request takes its tier from the provider entry's ``tier:`` # field — which is the simplest configuration and matches what # every provider in this example already declares. # # Operators with mixed posture (e.g., a single Bedrock model # under a ZDR addendum that should be Tier 2 instead of 3) use # ``overrides`` to call out the exception without restating the # whole provider's tier. inference_tiers: defaults: anthropic: 4 # standard commercial API openai: 4 vertex: 3 # routes through customer GCP project bedrock: 3 # routes through customer AWS account azure_openai: 3 # routes through customer Azure tenant cohere: 4 ollama: 1 # fully local — Mode 2 (air-gap-capable) vllm: 1 # fully local openai_compatible: 1 # local inference servers (vLLM, llama.cpp wrappers) # overrides: # anthropic-prod/claude-opus-4-7: 3 # if operator has a ZDR addendum # # Tier-1 placement for the Mode-2 aliases is inherited from # ``defaults.ollama: 1`` above — no override needed. The PRD §1.5.2 # framing for Tier 1 ("Local-only inference (air-gap-capable). No # outbound network is required. Customer data, prompts, and model # outputs never leave the deployment.") is what every Ollama-routed # request lands at; the routing-log row carries # ``routed_inference_tier=1`` and the chat UI surfaces the tier # badge accordingly. # ============================================================ # TIER POLICY — what the deployment allows # ============================================================ # The operator-level policy. Skills and Projects can require # stricter (higher minimum) tiers than this; they cannot # loosen the policy below allowed_tiers_global. tier_policy: # Tiers globally allowed in this deployment. # Requests at any other tier are refused with HTTP 403. allowed_tiers_global: [1, 2, 3, 4] # Tier 5 (consumer/free endpoints) is disallowed by default # Default minimum tier for non-privileged work. default_minimum_tier: 4 # Most cloud providers under standard accounts are Tier 4; # default of 4 means "use whatever's configured." # Minimum tier when project.privileged = true. privileged_minimum_tier: 3 # Privileged matters require enterprise ZDR or local # inference. For most-sensitive work, operators set this to 1. # Tiers that surface a warning in the chat UI. # The user can still proceed; the warning is informational. warn_on_tiers: [4, 5] # ============================================================ # RATE LIMITING # ============================================================ # Per-user, per-provider, per-model rate limits. # Backed by Redis token-bucket implementation. rate_limits: per_user: requests_per_minute: 60 tokens_per_minute: 100000 requests_per_day: 5000 per_provider: anthropic-prod: requests_per_minute: 200 tokens_per_minute: 500000 openai-prod: requests_per_minute: 200 tokens_per_minute: 500000 ollama-local: requests_per_minute: 30 # Local hardware constraint tokens_per_minute: 50000 burst_allowance: 1.5 # Multiplier for short bursts above sustained rate # ============================================================ # ANONYMIZATION (M2) # ============================================================ # Pre/post middleware that pseudonymizes sensitive entities # before the model call and rehydrates after. Bypassed for # Tier 1 (local) inference where there's no benefit. anonymization: enabled: false # M2 middleware shipped + runs; off by default — opt in per deployment # Apply when: # - The request's routed tier is in apply_at_tiers, AND # - The user has not opted out for the request, AND # - The chat is not in a privileged Project (privilege analysis # complications outweigh the marginal benefit) apply_at_tiers: [3, 4, 5] # Entity types to pseudonymize entity_types: - person_name - organization - email_address - phone_number - address - account_number - case_number - matter_number - financial_amount # Optional; can be too aggressive # Pseudonym format: ENTITY_TYPE_NNNN # e.g., PERSON_0001, ORG_0042, EMAIL_0007 # Mapping table is in-process memory only; never persisted # Rehydration: replace pseudonyms in response and citations # back to original values. Operates on response stream # before sending back to backend. # ============================================================ # CITATION ENGINE (PRD §3.3 / M2-C1) # ============================================================ # The api/ side runs the Citation Engine cascade (exact-match → # tolerant-match → LLM paraphrase judge → ensemble). Stage 3 # dispatches a judge call through this gateway; ``judge_model`` is # the alias the gateway resolves for that call. # # Defaults to ``fast`` so the judge runs on a smaller, cheaper # model than whatever generated the citation. Operators who want a # different judge model should pick one weaker than the # citation-generating model (the plan's calibration intuition: the # judge shouldn't be smarter than the author, only different). # Accepts either an alias from ``model_aliases:`` above or a fully- # qualified ``provider-name/model-name`` form. citation_engine: judge_model: fast # ---------------------------------------------------------------- # Stage 4 — ensemble verification (M2-D1) # ---------------------------------------------------------------- # When activated, the paraphrase judge runs in parallel across # ``judge_models`` and the verdicts aggregate under # ``aggregation_rule``. Activation is the OR of: # # 1. A skill's frontmatter ``ensemble_verification: true`` # 2. The chat's project ``ensemble_verification`` flag # 3. ``default_enabled`` below # # Replaces Stage 3 (single-judge) when activated. Cost-budget # pre-flight: the api/ estimates spend per message # (citations × n_judges × ~$0.005); if the estimate exceeds # ``max_cost_per_message_usd`` the cascade falls back to single- # judge Stage 3 with a warning logged. # # PRIVACY NOTE: each judge dispatch is an inference request that # routes through the gateway like any other. If judge_models # spans multiple provider tiers, the verification's privacy # envelope is the *weakest* (highest-numbered) tier across the # ensemble — exposed on ``message_citations.tier_envelope`` per # row so operators can audit ensemble exposure post-hoc. # # Default is fully opt-in: empty ``judge_models`` disables Stage 4 # even when ``default_enabled: true`` (defense in depth against # misconfiguration). ensemble_verification: default_enabled: false judge_models: [] # ``strict`` — every judge must verdict verified. Any miss → MISS. # ``majority`` — simple majority of verified verdicts wins; # even-N ties (e.g., 2-of-4) miss conservatively. Dissent # surfaces via the ``partial=true`` flag on the persisted row. aggregation_rule: strict max_cost_per_message_usd: 0.05 # # Example operator configuration (commented for safety): # # ensemble_verification: # default_enabled: true # judge_models: [fast, fast-anthropic, fast-vertex] # aggregation_rule: strict # max_cost_per_message_usd: 0.10 # ============================================================ # COST TRACKING # ============================================================ # Per-model cost rates for the cost tracker. Updated as # providers change pricing. Operator overrides allowed. cost_tracking: enabled: true # Cost per million tokens (input / output) # Update these when provider pricing changes rates: 'anthropic-prod/claude-opus-4-7': input_per_mtok: 15.00 output_per_mtok: 75.00 'anthropic-prod/claude-sonnet-4-6': input_per_mtok: 3.00 output_per_mtok: 15.00 'anthropic-prod/claude-haiku-4-5': input_per_mtok: 0.80 output_per_mtok: 4.00 'openai-prod/gpt-5': input_per_mtok: 1.25 output_per_mtok: 10.00 'openai-prod/gpt-5-mini': input_per_mtok: 0.25 output_per_mtok: 2.00 'openai-prod/gpt-5-nano': input_per_mtok: 0.05 output_per_mtok: 0.40 # Azure OpenAI — keyed by the Azure DEPLOYMENT-ID (the `models:` entries # on the azure-openai provider above), NOT the underlying model name. # Rates below mirror OpenAI GPT-5 list pricing; Azure enterprise # agreements (ZDR / BAA / committed-use) and newer point releases # (e.g. gpt-5.4) differ — confirm on the Azure OpenAI pricing page. 'azure-openai/gpt-5-prod': input_per_mtok: 1.25 output_per_mtok: 10.00 'azure-openai/gpt-5-mini-prod': input_per_mtok: 0.25 output_per_mtok: 2.00 'ollama-local/qwen3.5:9b': input_per_mtok: 0.0 # Local; no per-token cost output_per_mtok: 0.0 'ollama-local/qwen3.5:4b-nvfp4': input_per_mtok: 0.0 output_per_mtok: 0.0 'ollama-local/gemma4:e4b': input_per_mtok: 0.0 output_per_mtok: 0.0 # Embedding rates — OpenAI text-embedding-3-* charges only on # input tokens; output_per_mtok is 0 (no generation). The # embeddings success-row writer in the gateway uses these. 'openai-prod/text-embedding-3-small': input_per_mtok: 0.02 output_per_mtok: 0.0 'openai-prod/text-embedding-3-large': input_per_mtok: 0.13 output_per_mtok: 0.0 # Optional: budget alerts budget_alerts: daily_alert_threshold_usd: 50 weekly_alert_threshold_usd: 250 alert_email: ${BUDGET_ALERT_EMAIL:-admin@example.com} # ============================================================ # TELEMETRY # ============================================================ # Off by default. Operator opts in to specific sinks. telemetry: opentelemetry: enabled: ${OTEL_ENABLED:-false} endpoint: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4318} service_name: lq-ai-gateway service_version: ${LQ_AI_VERSION:-0.1.0} # The gateway emits traces for: every inference request # (with provider, model, tier, latency, cost), every # refusal (with reason), every fallback (with both # primary and fallback details). langfuse: enabled: ${LANGFUSE_ENABLED:-false} public_key_env: LANGFUSE_PUBLIC_KEY secret_key_env: LANGFUSE_SECRET_KEY host: ${LANGFUSE_HOST:-http://langfuse:3000} # Langfuse captures LLM-specific traces — full prompts, # full responses, eval scores. Self-hosted langfuse # keeps data in operator's environment. # ============================================================ # CIRCUIT BREAKER # ============================================================ # Provider-level failure handling. When a provider fails # repeatedly, the gateway routes to fallback and stops # attempting the failing provider for a cool-down period. circuit_breaker: enabled: true # Open the circuit after N consecutive failures failure_threshold: 5 # Stay open for this many seconds before allowing a probe recovery_timeout_seconds: 60 # When circuit is open, fall back per the model_aliases chain # ============================================================ # REQUEST VALIDATION # ============================================================ request_validation: # Reject requests with absurd parameters max_max_tokens: 16384 max_messages_per_request: 100 max_total_request_chars: 1000000 # ~250K tokens # Refuse requests above this token count without explicit # opt-in (prevents accidental large-context expensive calls) large_context_threshold_tokens: 50000 # ============================================================ # DEVELOPMENT-MODE OVERRIDES # ============================================================ # When running in dev mode, relax some safety checks. # Set LQ_AI_DEV_MODE=true to enable. dev_mode: enabled: ${LQ_AI_DEV_MODE:-false} bypass_rate_limits: false # Even in dev, keep rate limits log_full_prompts: false # PII concerns even in dev allow_tier_5: false # Tier 5 stays disallowed ``` ## `mcp.yaml.example` Reproduced in full from [`mcp.yaml.example`](https://github.com/LegalQuants/lq-ai/blob/671eab0d/mcp.yaml.example), comments included — the comments are the operator documentation. ```yaml # mcp.yaml — operator-allowlisted MCP servers (WS2 spec D2). Loaded by the # gateway (MCP_CONFIG_PATH; defaults to ./mcp.yaml). Each entry becomes a # type: mcp tool-provider behind the same egress boundary as gateway.yaml # tool_providers. The gateway is the sole egress + MCP-protocol speaker. mcp_servers: - name: example-mcp server_url: https://mcp.example.com/sse auth: none # none | bearer | oauth egress_tier: 2 allowlist: hosts: [mcp.example.com] # auth: bearer -> also set api_key_env: EXAMPLE_MCP_TOKEN # # auth: oauth -> per-user token supplied by the api at call time (PR4c). # oauth_client_id: lqai-acme # required: pre-registered public OAuth client_id # # for this MCP server's authorization server. # # The api reads this to initiate the out-of-band # # authorization-code flow on behalf of the user. # NOTE: the OAuth authorization-server host may differ from the MCP server # host (it is discovered at runtime from the AS metadata endpoint). Add it # to this server's allowlist.hosts so the gateway OAuth passthrough can # reach it. Example: if mcp.acme.example uses auth.acme.example as its AS, # set allowlist: {hosts: [mcp.acme.example, auth.acme.example]}. ``` ## Next - [Skill frontmatter](/lq-ai/reference/skill-frontmatter/) — the other generated schema. - [Install with Docker Compose](/lq-ai/operate/install-docker-compose/) — where these values are set. - [Rotate a leaked key](/lq-ai/operate/rotate-a-leaked-key/) — the provider-key settings in practice.