Bart Farrell: Flynn, welcome to KubeFM. For folks who don't know you, can you tell us about where you work and what you do?
Flynn: Happy to be here. Thank you for the welcome. I am a technical evangelist working at Buoyant on Linkerd. That job mostly means that I spend a lot of time figuring out how things actually work and then teaching people about it. But I am also heavily involved with Gateway API and the AI Gateway Working Group for the Cloud Native Computing Foundation. There's a lot of stuff that I do as well. That's dealing with how people actually use this stuff to get things done as opposed to how we work on Kubernetes itself.
Bart Farrell: That makes a lot of sense. I want to add a bonus question to this because Gateway API was a topic of conversations and even some controversy because of the NGINX deprecation, but looking at Gateway API, particularly in the AI context, for folks that aren't familiar with that, can you tell me about the things you're working on there and the value that it's providing?
Flynn: Gateway API itself is answering the question: If we look at how you manage traffic in a Kubernetes cluster, what commonalities can we identify? What things can we do so that you can learn them once and then be able to apply them to any sort of infrastructure that you have? This is stealing a concept from Kubernetes pretty directly where if you learn how a deployment works, you don't have to go relearn that to be able to use OpenShift or GKE or AKS. Deployments pretty much work the same everywhere. Gateway API is mostly looking at the problem: How do we do something similar for routing traffic, describing traffic, adding policy, enforcing policy on traffic, that sort of thing? How can we give people a common frame of reference that they can use? That has a lot of implications for a lot of different fields. For example, to the extent that we talk about policy these days, we may very well be talking about using Gateway API to describe traffic to which we want to apply a policy. In the AI world, being able to extend that sort of concept as we talk about selecting models is probably going to end up being important. We can extend the concept of routing to also talk about: What do we need to do computationally before we choose an endpoint for routing? What do we need to do to make AI safe, make inference safe? So it ends up showing up in a lot of far-flung corners of the Kubernetes cluster.
Bart Farrell: In a regulated Kubernetes environment, how far should you trust a service mesh as an enforcement layer for workload identity and authorization? If mTLS is automatic and policy is enforced at the proxy, what controls still need to exist outside the mesh to avoid creating a false sense of zero trust?
Flynn: I'm kind of smiling here because the answer to that question depends a lot on what kind of mesh your subject has a lot of background with. My background is with Linkerd, so I think you should trust it a very long way indeed. When we start talking about identity and policy enforcement and authentication, you get into a question: Do you want to be talking about identity? How do you want to be talking about identity? And in a lot of unregulated environments, you will see the network infrastructure, network metadata being used for identity. That doesn't work very well for zero trust, particularly not in heavily regulated environments, because one of the interesting things about Kubernetes is that we're kind of delegating control over the network and the associated metadata to the cluster itself. And it makes it very hard to say something like, this IP address is always this machine identity. This IP address is always this workload. Instead, it's a lot simpler to switch up to using cryptographic identities for that. In Linkerd, within a cluster, we rely on the fact that Kubernetes has cryptographic primitives associated with service accounts. So we can establish a strong cryptographic link between a workload and its associated service account. And then we can actually use that for mTLS certificates, which gives us a strong way that we can cryptographically associate identity with a workload at the mTLS level. And that means that we get to go through and institute policy that relies on your mTLS identity and get to trust that a very long way indeed.
Bart Farrell: Now a lot of regulated environments are designed around availability zones and fault isolation, but Kubernetes can still turn a local failure into a distributed one through retries, cross-zone traffic, or bad routing behavior. How would you design service-to-service traffic so that an AZ failure degrades cleanly instead of amplifying across the system?
Flynn: One of the really interesting things about that question is that it kind of doesn't depend on whether you're in a regulated environment or if you're not in a regulated environment. Good API design and good application design are kind of the same, whether or not the regulators are involved. To me, one of the first things you get to look at if you're trying to design for reliability ends up being: How much do you need the network for a given task anyway? Is there a way you can build your application so that if something fails and it can't talk to a given workload for a little bit, you can allow it to go back to its user with older data that you've flagged as older? Is it better to have it just fail and communicate the failure? What are your options here? You need things like retries and timeouts and circuit breaking, both to protect your workloads and to protect user agency. Retries, for example, are typically harder on the workload that you are adding retries for, because if something goes wrong, we'll immediately ask it again. Timeouts tend not to protect workloads at all, but they tend to arrange for your clients to still be able to have agency. They can decide what to do with a problem instead of just waiting 10 minutes and having nothing happen. But you end up with questions like: If you are in a timeout situation, do you need to immediately go back and tell the user something has gone horribly wrong? Or is it okay to show them older data or cached data? Is it okay to switch over to some backup service? These are all questions that you need to address in the design of the application. And you can only do it by thinking carefully about what your users need and how you should respond to them in the case of failures. This is actually something of a common thread of mine. I've spent most of the last decade arguing that in the cloud-native world, we should start by thinking about how the humans will be interacting with things and using things rather than thinking about what's easy to implement. It's really easy to just slap a circuit breaker or a timeout on something. It's much harder to think about how that's going to affect the people using your application, but it is much more important.
Bart Farrell: And your recent gRPC work showed that one seemingly small protocol declaration can silently remove routing, metrics, retries, canaries, and even authorization policy. In a bank, how do you validate that the controls you think are being enforced by the mesh are actually being enforced, especially as Kubernetes, Gateway API, and application protocols evolve independently?
Flynn: That was a deep dive into a particular conformance failure for a Gateway API test where the root cause was that it had declared that a given service was using a protocol that Linkerd decided must mean was opaque because it was a nonstandard value. And you're right, that absolutely does open the door: you have everything set up and then a developer changes a protocol value and suddenly things get turned off that you're not expecting to be turned off. This kind of gets into an interesting distinction that I think the cloud-native world is only just starting to grapple with, which is the distinction between technical policy and organizational policy. In this particular case, the technical policy did exactly what it was supposed to do. Somebody declared that an app protocol was not gRPC, and so we turned off all the gRPC functionality. But organizationally, that could very much run afoul of the organizational policy that a given set of traffic has to be properly authenticated, it has to be properly load balanced, etc. Organizational policies, to my mind, are usually things that are best served with audit or with specific policy enforcement engines. For example, there are a number of ways that we can have a Kubernetes cluster validate that when you have applied a resource, you've applied it following certain rules. The Validating Admission Policy that was relatively recently added to Kubernetes is a simple example of that, but it can be a very effective example of that. It wouldn't be hard to write a Validating Admission Policy that says that everything in this namespace has to have that protocol HTTP/2 or kubernetes.io/h2c, if you want to get technical. It's also the case that we have policy enforcement engines like Kyverno and its ilk that can go after the fact and do a much more sophisticated set of matching to see whether a given policy is doing the right thing. In the world of regulated environments, you probably are going to end up using a mixture of both. But it's going to be very important, I think, to recognize that the technical policy and the organizational policy can be very different. And so arranging it so that you are setting up the policy engine to enforce organizational policy is important and is very different from setting up the Kubernetes cluster to follow the technical policies that people put in place.
Bart Farrell: Flynn, you're working on a lot of different things. What's next for you?
Flynn: There are a couple of things immediately. And some of these have to do with what we're talking about. And some of these have absolutely nothing to do with what we're talking about, which is always kind of funny. Gateway API is coming up on a release. I am currently involved in getting my elbows dirty, making sure of what's supposed to be in the release, what's not going to make it, things like that.
Bart Farrell: No pressure.
Flynn: I've landed in the role of helping manage Gateway API releases for the last several releases, which is always a little bit amusing to me. I've been doing a lot of work recently on an escape room because at KubeCon in Salt Lake City, Buoyant is reprising the escape room party that we did. So that's been a lot of fun. And it's always entertaining to take a step back from designing things to actually trying to use them. And I would recommend that everybody who works on architecture or design should have to use the things you're designing on a regular basis because it's kind of the only way to stay in touch with the impact that your design work has and to get a sense of whether you've been designing things that work well for users or whether you're designing things that are adding more friction than they need to for your users. So that's kind of fun. There's also a lot of work going on in the AI Gateway Working Group around looking at inference making its way into applications and figuring out: How do we make this usable? How do we make it safe? And how does that affect Kubernetes as a whole? There's a lot of stuff going on.
Bart Farrell: Sounds like it. And if people want to get in touch with you, what's the best way to do that?
Flynn: The easiest way is to go to your favorite cloud-native-related Slack and find Flynn as a username. You can also send me email at flynn@buoyant.io. That will work fine as well. But Slack might be faster.
Bart Farrell: I would recommend Slack. Flynn, great talking to you and looking forward to all the next steps that you're taking in the working group, as well as that killer escape room you're going to be setting up for KubeCon.
Flynn: Thanks much. It's been a pleasure to be here. We'll do it again sometime.
