← All recipes

Recipe: mutual TLS between the gateway and the backend

1. Problem

TLS at the edge protects traffic between the browser and the gateway. But inside the cluster, the gateway-to-backend hop is often plain HTTP on a "trusted" network — and "trusted network" is exactly the assumption that lets an attacker who gets a foothold move laterally. If anything on the network can call the backend, the backend has no way to know it is really talking to the gateway.

2. Concept

Mutual TLS: both sides present a certificate and both verify the other's against a shared CA. The backend accepts a connection only from a client holding a cert the CA signed (the gateway); the client accepts only a backend whose cert the CA signed. Neither trusts the network — they trust the certificates. It is the concrete form of "never trust the client," applied service-to-service.

3. Implementation

The Go backend runs an optional mTLS listener (backends/go/cmd/server/main.go), enabled by MTLS_ENABLED:

mtlsSrv := &http.Server{
    Addr:    cfg.MTLSAddr, // :8443
    Handler: handler,
    TLSConfig: &tls.Config{
        MinVersion: tls.VersionTLS12,
        ClientAuth: tls.RequireAndVerifyClientCert, // the crux
        ClientCAs:  pool,                            // our CA
    },
}
mtlsSrv.ListenAndServeTLS(cfg.MTLSCertFile, cfg.MTLSKeyFile)

Certs are generated by infra/tls/gen-certs.sh (a dev CA, a backend server cert, a gateway client cert) into infra/tls/certs/, which is gitignored — private keys never enter the repo. The demo overlay infra/tls/docker-compose.mtls.yml mounts them and turns the listener on.

In production the gateway presents the client cert via a Traefik serversTransport:

http:
  serversTransports:
    backend-mtls:
      rootCAs: ["/certs/ca.crt"]
      certificates:
        - certFile: /certs/client.crt
          keyFile: /certs/client.key
  services:
    active-backend:
      loadBalancer:
        serversTransport: backend-mtls
        servers: [{ url: "https://backend:8443" }]

4. How to see it working

infra/tls/gen-certs.sh
COMPOSE_PROFILES=go docker compose -f docker-compose.yml -f infra/tls/docker-compose.mtls.yml up -d

C=infra/tls/certs
curl --cacert $C/ca.crt --cert $C/client.crt --key $C/client.key https://localhost:8443/health   # 200
curl --cacert $C/ca.crt https://localhost:8443/health                                             # handshake rejected

Verified: with the client cert, 200; without it, the TLS handshake fails (HTTP 000). The backend refuses to speak to anyone who cannot prove they are the gateway.

5. Trade-offs

  • Certificate lifecycle is the hard part. Issuing, rotating, and revoking certs at scale is a job for a mesh (Istio, Linkerd) or SPIFFE/cert-manager, not hand-rolled openssl. This recipe is the mechanism; the operations are a platform concern.
  • Only the Go backend implements it here — it is the worked example the spec asks for. The pattern is identical for the others (an mTLS listener + the gateway transport), left out to avoid seven copies.
  • mTLS authenticates the service, not the user. It says "the gateway called me," not "this end user is allowed" — the JWT still does that. They are different layers and you want both.