A. B. Khan

Multi-cluster mesh for data residency — residency is a topology decision, not a config flag

The Mesh Series · Article 39 · Part G

Multi-cluster & residencyCompliance evidenceSPIFFEAmbient meshIstioLGTM stackmTLSDORA

Cover: Multi-cluster mesh for data residency

Article 38 ended on a problem that sounds administrative and is not. Your Indian arm's CERT‑In report is due in six hours. The evidence that proves it — the access‑log lines, the signed aware‑at record, the Loki result set — must stay in India for 180 days. Your group's DORA report to the FMA wants the same evidence for the same incident. The two regulators are not arguing about the facts. They are arguing about where the facts are allowed to sit.

You can't fix that with a label on a namespace. You fix it with the shape of the mesh.

One idea: residency is a topology decision, not a config flag

Most residency conversations start with storage. Where is the database, where is the bucket, which Azure region. Those are the easy half. The hard half is the identity plane and the evidence plane, because both of them are produced by the mesh, and a mesh has exactly one shape: either two clusters share a trust domain or they do not.

Share it, and a workload in Pune can present a SPIFFE identity that a workload in Vienna will accept on sight, because both chain to the same root. That is enormously convenient. It is also a single root of trust whose private key physically lives in one jurisdiction, and a single blast radius that spans two.

Don't share it, and the two clusters are strangers. Nothing crosses without an explicit gateway, an explicit trust bundle exchange, and an explicit policy. Residency becomes a property of the architecture rather than a promise in a document.

The contrarian take: for a regulated group, the second shape is the default, and the first shape is the exception you justify. The Istio documentation walks you towards a single mesh across clusters because that is what the multi‑cluster feature is for. For a group with an in‑country obligation on one side and a DORA obligation on the other, "one mesh" is a design decision with a legal shadow, and you should make it on purpose.

Diagram: residency

Two clusters, two trust domains, two offline roots, two evidence stores that never replicate across the line. The only thing that crosses the middle is an east‑west gateway carrying mTLS for services you have named by hand.

Why it matters: a shared trust domain is a shared blast radius

Three consequences of the single‑mesh shape, in the order a regulator will find them.

The root lives somewhere. In a multi‑primary mesh both clusters' istiod instances sign workload certificates from intermediates that chain to one root. That root key is a file. It is in a Key Vault, or an HSM, or — in too many clusters — a Kubernetes secret in istio-system. Whichever it is, it has a region. If the Indian cluster's workload identities chain to a root held in West Europe, then the credential that authenticates every Indian payment service is derived from key material outside India. You will be asked about that. Article 40 covers keeping the root offline; this article is about which jurisdiction's floor it is offline in.

Policy composes across the line whether you meant it to or not. An AuthorizationPolicy names principals as SPIFFE IDs: cluster.local/ns/payments/sa/kyc. In a shared trust domain, that string matches the kyc service account in either cluster. Write a rule that allows kyc to call ledger in Vienna, and the Pune kyc can call it too, with a valid certificate, through the east‑west gateway, with nothing in the log to say the call crossed a border except an IP you are not retaining. A cross‑border allow is not something you configure in that model. It is something you fail to prevent.

Evidence follows the identity. The access logs, traces and signed packs from Articles 24–27 are keyed on the SPIFFE identity. If identity is one namespace across two countries, the natural thing — the thing every default does — is one Loki, one Tempo, one evidence bucket. CERT‑In's 180‑day in‑country retention and RBI's payment‑data localisation say the Indian arm's records stay in India. SAMA's in‑Kingdom expectations say the same for a Saudi entity. DORA's Article 12(3) wants restoration tested on segregated systems. A single evidence store in one region satisfies at most one of those at a time.

The pattern beneath all three: the single‑mesh shape makes sharing the path of least resistance, and residency is the discipline of not sharing by default. Put the boundary in the topology and the defaults start working for you.

How it comes together: two trust domains, one named bridge

The shape has four parts, and each one is a place where the convenient default is the wrong one.

Separate trust domains, separate roots. Each cluster gets its own trustDomain — at.group.internal, in.group.internal — and its own plug‑in CA via the cacerts secret, chained to a root that is generated and stored in that jurisdiction and never leaves it. The SPIFFE IDs are now different strings: spiffe://in.group.internal/ns/payments/sa/kyc cannot be confused with its Austrian twin, and an AuthorizationPolicy that names one does not accidentally admit the other.

Trust bundles exchanged, not merged. For the handful of calls that legitimately cross — a group‑level reporting service reading aggregated, non‑personal figures, say — each cluster is given the other's root certificate as an additional trust anchor. Istio supports this through the meshConfig.trustDomainAliases list and a trust bundle; SPIRE does it natively through federation. The important property: trusting the other root to authenticate is not the same as authorising anything. You still write the allow rule, and you write it against the foreign trust domain explicitly.

An east‑west gateway that carries only mTLS, for only named services. The sidecar multi‑primary multi‑network pattern — the mature, GA path in 1.31 — routes cross‑cluster traffic through a dedicated gateway with ISTIO_META_ROUTER_MODE=sni-dnat, and only services you have marked as exported are reachable through it. Everything else in the cluster is invisible from the other side. Ambient's equivalent is beta since 1.29 and carries meshed mTLS only; it cannot expose istiod, which is a feature.

Evidence stays home. Each cluster has its own Loki, Tempo and WORM bucket in its own region, with the Article 27 signing key in a Key Vault in that same region. The cross‑border reporting path reads conclusions — a signed summary, a count, a verdict — never the raw records. That is the difference between "the FMA report cites the Indian incident" and "the Indian access logs are in Frankfurt."

A note on the ambient‑vs‑sidecar question, because it decides the code beat below. Ambient multi‑cluster in 1.31 is multi‑primary only, waypoints must be identically named in every cluster, and only the local cluster's service‑scope configuration is honoured — a remote istio.io/global label is ignored. Those are workable rules, but they are beta rules, and they are precisely the kind of thing a regulator asks you to defend. For the cross‑border bridge, use the GA sidecar pattern until ambient multi‑cluster is GA; keep ambient inside each cluster where Article 32 already put it.

Show the config: install with separate roots, then allow by named trust domain

First the install. Note what is deliberately not shared: the trust domain, the cluster name, the network, and the root. The one thing that is shared is meshID, which lets the two control planes recognise each other as peers once you hand them remote secrets.

Code: install

Now the policy on the Austrian side. The reports service may be called by the Austrian ledger and by exactly one Indian service, named by its foreign trust domain. The Indian kyc service — same service‑account name, different country — gets nothing, because its SPIFFE ID is a different string.

Code: authzpolicy

Read the second rule the way an auditor will: a foreign identity, a single method, a path prefix that returns aggregates. That line is your cross‑border data transfer, written down.

Security · Audit · Integration

Security: two roots, two jurisdictions, no shared private key; a compromise of one cluster's CA mints identities that the other cluster will authenticate only if you listed its root, and authorise only where you named the principal.

Audit: the cross‑border allow is a single AuthorizationPolicy rule naming a foreign trust domain, a method and a path — one line to show, one access log to prove it held; the raw evidence for each entity never leaves its region, so CERT‑In's 180‑day in‑country retention and a DORA restoration test on segregated systems are both satisfied by the same shape.

Integration: meshID shared; trustDomain, clusterName, network, cacerts, Loki, Tempo and the WORM bucket all per cluster; Entra remains the single user‑identity plane across both (Article 10's composition still holds — Entra knows who, each mesh knows what, in its own country).

Alternatives, and when you'd pick them

One mesh, one trust domain, two clusters. The documented default, and the right answer for a single legal entity with two regions for availability — an Austrian bank with Vienna and Frankfurt clusters, say, under one regulator. You get transparent failover and one policy model. Pick it when the clusters share a regulator, a data‑residency rule and an incident clock. Do not pick it across a residency line and then try to add the line back with namespace labels.

Primary‑remote. One control plane, remote clusters run only data planes. Cheaper to run, but the remote cluster's identities are minted by a control plane in another jurisdiction, and if the link drops the remote cannot issue certificates. For residency this inverts the requirement. Ambient does not support it anyway.

No east‑west at all — two meshes, an API gateway between them. The purest residency story: the two clusters are simply two systems, and the cross‑border call is a normal north‑south API call through a gateway with its own JWT validation. Pick this when the cross‑border traffic is a handful of coarse calls and you would rather explain an API contract to a regulator than a trust bundle. The cost is losing mesh‑native identity on the bridge; you get Entra identity instead, which Article 10 would argue is the right identity for a cross‑entity call anyway.

SPIRE federation instead of Istio trust‑bundle exchange. If you already run SPIRE as the identity provider (the direction the Evidence Series lab takes), federation is first‑class: each trust domain publishes a bundle endpoint, the other fetches it on a schedule, and the policy model is identical to the above. Pick it when SPIRE is already the CA; do not adopt SPIRE for this alone.

From simple to standard, and what's next

The simple version is one mesh across two regions with a residency paragraph in the architecture document. The standard version is two trust domains with roots generated and kept in their own jurisdictions, an east‑west gateway that carries mTLS for named services only, one explicit cross‑border allow rule per legitimate flow written against the foreign trust domain, and an evidence stack per region whose raw records never replicate across the line. Nothing in it is exotic. It is Article 14's identity, Article 16's default‑deny and Article 27's evidence, drawn on a map.

A map has a failure mode: one of the two clusters goes away. The root of trust you kept offline in one jurisdiction now has to come back, in order, before the identities, before the policy, before the workloads, before anyone can prove anything. Next, Article 40: disaster recovery for a regulated mesh — recovering the proof, not just the app.

Residency is not a flag you set. It is a line you draw, and the mesh is the pen.


Next in the series: Art. 40, disaster recovery for a regulated mesh (Part G). Earlier: Art. 38 (four confession clocks — why the evidence can't travel), Art. 32 (sidecar → ambient), Art. 30 (regional frameworks and residency), Art. 14 (mTLS and SPIFFE), Art. 16 (default‑deny), Art. 27 (signed evidence).

Key skills & topics in this article

Multi-cluster & residencyCompliance evidenceSPIFFEAmbient meshIstioLGTM stackmTLSDORA

  • #Istio
  • #Kubernetes
  • #DataResidency
  • #ServiceMesh
  • #SPIFFE
  • #DORA
  • #RBI
  • #SAMA
  • #ZeroTrust
  • #RegTech

Also on LinkedIn.