A. B. Khan

Defense in depth — validate at the mesh and in the code

The Mesh Series · Article 13

mTLSAmbient meshSPIFFE

Cover: Defense in depth

The Mesh Series · Article 13 · Where Entra meets Istio

Article 11 validated the user token at the mesh edge. Article 12 validated it again, deeply, inside the service. Article 12 closed on the question you've been owed an answer to: if the gateway already checked it, why check it again? Isn't the second pass redundant?

It isn't, and the reason is the whole discipline of defense in depth: no single control is the control. Each layer exists to catch a failure the others can't — and, more importantly, to still be standing when another layer silently isn't.

Diagram: layers

Two gates on the same token, and neither trusts the other to have done the job. That distrust is the feature. It only pays off, though, if the two layers fail independently — which is the test you apply to decide whether a second layer is real defence or just extra config to get wrong.

Why regulated enterprises should care

Two real failures make the abstract concrete, and they're the two ways a single gate lets you down.

A gate can fail open. Picture the edge configured exactly as Article 11 warned: RequestAuthentication validating tokens, but the AuthorizationPolicy that actually rejects the tokenless ones missing. The gateway is present, it looks like security in the diagram, and it enforces nothing — until infosec flags it for allowing open access. Now ask the question that matters: when that gate failed open, what stopped an anonymous request from reaching the data? If the service validates independently, the answer is "the service did." If the edge was your only gate, the answer is "nothing did, for however long it took someone to notice."

A gate can fail invisibly. A signing-key rotation breaks token validation, but it doesn't announce itself as a clean error — it surfaces as a generic exception, a network wobble, auth failing across the board with no deploy to blame. While you're tracing it by timing, the question again is what's still holding. A second, independent layer is what buys you the room to diagnose without being wide open in the meantime.

For a regulated institution this isn't philosophy — it's an explicit expectation. CERT-In's incident regime, SAMA's maturity model, ISO 27001, PCI-DSS: every one of them assumes layered controls, because every one of them has watched a single control fail. "We validate at the gateway" is one control. "We validate at the gateway and independently in every service, and the workload identity is enforced on the wire besides" is a posture. The auditor is asking for the second sentence.

How the pieces come together

Three independent axes, each catching something the others structurally cannot.

The edge (Article 11) is coarse and cheap: it drops the anonymous and the malformed before they cost you a single service invocation, and it gives you the one provable chokepoint an auditor can inspect. The service (Article 12) is deep and specific: it re-checks signature, issuer, audience, and lifetime at the point that actually touches data, so an edge bypass or a confused-deputy token dies here. The mesh mTLS layer — the workload plane, next article — is orthogonal to both: it enforces which workload is even allowed to open the connection, regardless of what token rides inside it. A stolen user token still can't traverse the graph from the wrong service.

That orthogonality is the point, and it's the test for whether a layer earns its place: each of these catches a different failure mode. Stack two controls that catch the same failure and you haven't added defence — you've added a second thing to misconfigure. Real defence in depth is layers that fail independently; theatre is the same check twice.

Now the honest tension, because this series doesn't sell inevitability. Under Istio's sidecar model, that per-hop L7 validation carries real cost — a proxy on every pod. Ambient mode changes the arithmetic: ztunnel makes the L4 mTLS layer nearly free, but the L7 edge validation moves into a waypoint proxy, so you get fewer proxies, not none. That migration is Article 32. The layering principle survives it unchanged; only where each layer runs moves.

Show the config

There's little new code here — that's the point. Defense in depth is the composition of the two gates you already built, so the artifact is the pair standing together.

Code: two gates

And the one question worth settling before you're forced to answer it live: if you could keep only one, which? Keep the service. The edge is a cheap optimisation and your provable boundary, but it is bypassable — an internal call, a misrouted ingress, a future refactor. The service validation is the layer that's still true when the topology changes underneath it. You keep both, always. But knowing which one is load-bearing tells you where to spend your care.

Security · Audit · Integration

Security property gained. Independent failure. The system's security no longer rests on any one control being correctly configured and healthy at all times — it rests on all of them failing simultaneously, which is a far taller order for an attacker or an outage.

Audit evidence produced. You can show layered controls that map cleanly onto framework language: a network/identity boundary at the edge, independent verification in the workload, and workload-identity enforcement on the wire. "Show me your compensating controls" has a concrete answer instead of a shrug.

How it integrates. This article is the hinge of Part B: it names why the edge (11) and the service (12) coexist, and it sets up the third axis — the workload plane's own gate. mTLS and SPIFFE, the layer that doesn't care what token you carry because it's checking what you are, is next.

Alternatives, and when you'd actually pick them

Single layer — edge only. Cheap, and genuinely fine for a low-stakes internal service where a bypass is implausible and the data isn't regulated. The failure mode is exactly the "fail open" story above: one misconfiguration and there's no backstop. Choose it with eyes open, not by default.

Single layer — service only. More defensible than edge-only, because the service layer survives topology changes. What you lose is the provable chokepoint — you can no longer point to one place and demonstrate that nothing anonymous gets in, so you're back to attesting across every service. Reasonable before you've adopted a mesh; weaker once you have.

Defense-in-depth theatre. The anti-pattern to name out loud: stacking controls that catch the same failure and calling it layering. Two JWT validations with identical parameters in the same trust zone don't add a layer — they add a maintenance surface and a false sense of depth. The test is always the same: does this control fail independently of the others, and does it catch something they don't? If not, it's cost wearing the costume of safety.

The way I've actually watched depth matter had nothing to do with an attacker. Environments that were supposed to be isolated weren't — a development system could still reach production — and dev workloads quietly generated data straight into the production store. No breach, no alarm, just corrupt data accumulating because the one boundary everyone assumed was there wasn't, and there was no second control behind it to notice traffic that should never have existed. That's the argument for depth stripped of drama: a single control you trust is a single control that can be silently wrong, and the second layer is what turns a silent corruption into a request something actually stops.

From the toy to the standard

The toy is two gates on one namespace. The industry-standard version is layered by default everywhere it matters: the edge drops the anonymous cheaply and provably, every service validates independently at the point of data access, the mesh enforces workload identity on the wire beneath both, and each layer is chosen because it catches a failure the others can't — never duplicated for the comfort of a second identical check. When one layer fails open or fails silent, another is still standing, and you have the room to fix it without being exposed.

We've now leaned three times on a layer we haven't actually built: the workload plane, mTLS, the identity that rides on the connection itself rather than in a token. It's time to open it up — how a workload gets a cryptographic identity, how two services prove themselves to each other with no shared secret, and how those certificates rotate without anyone touching them.

That's Article 14: mTLS and SPIFFE.


Next in the series — Article 14: mTLS & SPIFFE.

Key skills & topics in this article

mTLSAmbient meshSPIFFE

  • #ServiceMesh
  • #Istio
  • #Azure
  • #dotnet
  • #ZeroTrust
  • #Kubernetes

Also on LinkedIn.