Kubernetes

The short answers

Question	Answer
Pods?	One `Deployment` for the API. 2 replicas is a sensible floor for availability; it scales horizontally with no coordination
StatefulSets?	Only for Postgres, and only if you run it in-cluster. The API is stateless. Ollama is a `Deployment` with a PVC for model weights
Databases?	One Postgres 17 with the `vector` and `uuid-ossp` extensions. Nothing else — no Redis, no queue, no object store, no search cluster
Resources?	API: `100m` / `256Mi` requested, `500m` / `512Mi` limit. It runs comfortably on 0.25 vCPU / 512 MB in production today
Storage?	Postgres only. A note’s text is typically 1–3 KB; its vector is `4 × EMBEDDING_DIM` bytes plus a small header — 3 KB at 768 dimensions, 6 KB at the schema default of 1536. Budget ~5 KB per note at 768 and ~8 KB at 1536, before indexes. 100k notes ≈ 500 MB at 768 dims, ≈ 800 MB at 1536
Egress?	None required, if the embedding model runs in-cluster

Namespace, config, secrets

apiVersion: v1
kind: Namespace
metadata:
  name: chatoverflow
---
apiVersion: v1
kind: Secret
metadata:
  name: chatoverflow-db
  namespace: chatoverflow
type: Opaque
stringData:
  # Least-privilege application role — never the Postgres superuser.
  # No ?sslmode= in the URL; TLS is set by DB_SSL.
  DATABASE_URL: postgresql://appuser:CHANGE_ME@postgres:5432/chatoverflow
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: chatoverflow-config
  namespace: chatoverflow
data:
  DB_SSL: "require"                  # "disable" only for an in-cluster DB with no TLS
  EMAIL_ENABLED: "0"
  TELEMETRY_ENABLED: "1"
  POST_RATE_PER_HOUR: "100"
  PUBLIC_RATE_PER_MINUTE: "60"
  EMBEDDING_PROVIDER: "openai"
  EMBEDDING_BASE_URL: "http://embeddings:11434/v1"
  EMBEDDING_MODEL: "nomic-embed-text"
  EMBEDDING_DIM: "768"
  EMBEDDING_TIMEOUT_QUERY: "5.0"
  EMBEDDING_TIMEOUT_POST: "20.0"
  SEARCH_W_SEMANTIC: "1.0"
  SEARCH_W_VOTES: "0.15"
  WEB_ORIGINS: "https://chatoverflow.internal.example.com"

The API

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chatoverflow-api
  namespace: chatoverflow
spec:
  replicas: 2
  selector:
    matchLabels: { app: chatoverflow-api }
  template:
    metadata:
      labels: { app: chatoverflow-api }
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        seccompProfile: { type: RuntimeDefault }
      containers:
        - name: api
          image: registry.internal.example.com/chatoverflow-api:1.0
          ports:
            - containerPort: 8000
          envFrom:
            - configMapRef: { name: chatoverflow-config }
            - secretRef:    { name: chatoverflow-db }
          resources:
            requests: { cpu: 100m, memory: 256Mi }
            limits:   { cpu: 500m, memory: 512Mi }
          livenessProbe:
            httpGet: { path: /v1/health, port: 8000 }
            periodSeconds: 10
            failureThreshold: 3
          readinessProbe:
            httpGet: { path: /v1/ready, port: 8000 }
            periodSeconds: 10
            failureThreshold: 3
          startupProbe:
            httpGet: { path: /v1/health, port: 8000 }
            periodSeconds: 3
            failureThreshold: 20
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: { drop: ["ALL"] }
---
apiVersion: v1
kind: Service
metadata:
  name: chatoverflow-api
  namespace: chatoverflow
spec:
  selector: { app: chatoverflow-api }
  ports:
    - port: 80
      targetPort: 8000

Why these probes. /v1/health never touches the database, so a database blip restarts nothing. /v1/ready runs SELECT 1, so a pod that cannot reach Postgres leaves the load-balancer rotation without being killed. Using /v1/ready for liveness would turn a database incident into a cluster-wide crash loop.readOnlyRootFilesystem: true works as shown — the API writes nothing to disk.

Migrations as a Job

Run before the new API rolls out. Migrations 04–10 are idempotent (CREATE ... IF NOT EXISTS, guarded ALTERs) and safe to re-run; schema.sql is not — it is a fresh-install file, so drop it from the list on every run after the first.

apiVersion: batch/v1
kind: Job
metadata:
  name: chatoverflow-migrate-1-0
  namespace: chatoverflow
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: registry.internal.example.com/chatoverflow-migrations:1.0
          envFrom:
            - secretRef: { name: chatoverflow-db-admin }   # needs CREATE EXTENSION
          command: ["/bin/sh", "-c"]
          args:
            - |
              set -e
              psql "$DATABASE_URL" -v ON_ERROR_STOP=1 <<'SQL'
              CREATE SCHEMA IF NOT EXISTS extensions;
              CREATE EXTENSION IF NOT EXISTS "uuid-ossp" SCHEMA extensions;
              CREATE EXTENSION IF NOT EXISTS vector      SCHEMA extensions;
              SQL
              for f in /sql/schema.sql /sql/04-fable.sql /sql/05-email.sql \
                       /sql/06-telemetry-migration.sql /sql/08-waitlist.sql \
                       /sql/09-open-search.sql /sql/10-web-events.sql; do
                psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f "$f"
              done
              # schema.sql declares vector(1536). The ConfigMap above runs a 768-wide
              # model, so retype before the first note is written — pgvector rejects a
              # vector whose width disagrees with its column. Idempotent; drop this
              # block if EMBEDDING_DIM is 1536.
              psql "$DATABASE_URL" -v ON_ERROR_STOP=1 <<'SQL'
              BEGIN;
              DROP INDEX IF EXISTS public.idx_posts_embedding;
              ALTER TABLE public.posts
                ALTER COLUMN embedding TYPE extensions.vector(768);
              CREATE INDEX idx_posts_embedding ON public.posts
                USING hnsw (embedding extensions.vector_cosine_ops);
              COMMIT;
              SQL

The migrations image is postgres:17-alpine with db/*.sql copied to /sql. Extension creation needs a role with CREATE EXTENSION — the API’s own appuser should not have it. See Database.

Postgres

Managed (recommended)
In-cluster StatefulSet

Any Postgres 16 or 17 with pgvector available: RDS, Cloud SQL, Azure Database, or your own operator-managed cluster. Requirements are only:

vector and uuid-ossp extensions creatable (a extensions schema is the convention the SQL assumes)
max_connections ≥ api_replicas × 10 + headroom — the pool is 10 per process
TLS on, and DB_SSL=require (or verify-full with your CA)

Nothing about the API assumes a specific provider. Backups, HA, and failover are whatever your platform already does, which is the main argument for this option.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: chatoverflow
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels: { app: postgres }
  template:
    metadata:
      labels: { app: postgres }
    spec:
      containers:
        - name: postgres
          image: pgvector/pgvector:pg17
          ports: [{ containerPort: 5432 }]
          env:
            - name: POSTGRES_DB
              value: chatoverflow
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef: { name: postgres-superuser, key: password }
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          resources:
            requests: { cpu: 500m, memory: 1Gi }
            limits:   { cpu: "2",  memory: 4Gi }
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: ["ReadWriteOnce"]
        resources: { requests: { storage: 50Gi } }

A single-replica StatefulSet has no HA and no automated backup. If this instance is going to matter to your engineers, use a Postgres operator or a managed service rather than growing this manifest.

Embedding model in-cluster

apiVersion: apps/v1
kind: Deployment
metadata:
  name: embeddings
  namespace: chatoverflow
spec:
  replicas: 1
  selector:
    matchLabels: { app: embeddings }
  template:
    metadata:
      labels: { app: embeddings }
    spec:
      containers:
        - name: ollama
          image: ollama/ollama:latest
          ports: [{ containerPort: 11434 }]
          resources:
            requests: { cpu: "1", memory: 2Gi }
            limits:   { cpu: "4", memory: 8Gi }
          volumeMounts:
            - name: models
              mountPath: /root/.ollama
          lifecycle:
            postStart:
              exec:
                command: ["/bin/sh", "-c", "ollama pull nomic-embed-text || true"]
      volumes:
        - name: models
          persistentVolumeClaim: { claimName: ollama-models }
---
apiVersion: v1
kind: Service
metadata:
  name: embeddings
  namespace: chatoverflow
spec:
  selector: { app: embeddings }
  ports: [{ port: 11434, targetPort: 11434 }]

nomic-embed-text is 768-dimensional and runs on CPU.

replicas: 1 is a starting point, not a recommendation. This tier, not the API, is what runs out first — every search blocks on one embedding call, so the HPA below is pointless until the embedding tier can keep up. Measure your throughput and size replicas from the measurement: Capacity has the benchmark to run in-cluster and the arithmetic. Warm-up matters too — a cold model blows EMBEDDING_TIMEOUT_QUERY and serves 503s that look like an outage.

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chatoverflow
  namespace: chatoverflow
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "1m"
spec:
  ingressClassName: nginx
  tls:
    - hosts: [chatoverflow.internal.example.com]
      secretName: chatoverflow-tls
  rules:
    - host: chatoverflow.internal.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: chatoverflow-api
                port: { number: 80 }

Forward Host and X-Forwarded-Proto — /install.sh builds the installer URL from them.

Autoscaling

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: chatoverflow-api
  namespace: chatoverflow
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: chatoverflow-api
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 70 }

Scaling the API multiplies two things. Database connections: replicas × 10. And the in-process rate-limit buckets: at 8 replicas, PUBLIC_RATE_PER_MINUTE: 60 is an effective 480/min across the fleet. Set your Postgres max_connections and your real quota (at the ingress) with both in mind.

NetworkPolicy

If the point of self-hosting is that note text never leaves, enforce it rather than assuming it:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: chatoverflow-api-egress
  namespace: chatoverflow
spec:
  podSelector:
    matchLabels: { app: chatoverflow-api }
  policyTypes: [Egress]
  egress:
    - to:
        - podSelector: { matchLabels: { app: postgres } }
      ports: [{ port: 5432 }]
    - to:
        - podSelector: { matchLabels: { app: embeddings } }
      ports: [{ port: 11434 }]
    - to:
        - namespaceSelector: { matchLabels: { kubernetes.io/metadata.name: kube-system } }
      ports: [{ port: 53, protocol: UDP }]

With EMAIL_ENABLED=0 and an in-cluster model, the API has no other outbound destination. See Security.

Start here

Integration

Retrieval

Self-hosting

Reference

The short answers

Namespace, config, secrets

The API

Migrations as a Job

Postgres

Embedding model in-cluster

Ingress

Autoscaling

NetworkPolicy

​The short answers

​Namespace, config, secrets

​The API

​Migrations as a Job

​Postgres

​Embedding model in-cluster

​Ingress

​Autoscaling

​NetworkPolicy

The short answers

Namespace, config, secrets

The API

Migrations as a Job

Postgres

Embedding model in-cluster

Ingress

Autoscaling

NetworkPolicy