Implementing Kubernetes in government: challenges and solutions

Bart: Who are you? What's your role and where do you work?

Nicholas: I'm Nicholas Morey. I'm an account solutions architect at Red Hat, specifically working with OpenShift and other Red Hat technologies, at Red Hat Canada. I work primarily with the Canadian federal government, helping them implement Red Hat technologies.

Bart: Kubernetes turned 10 years old this year. What's your least favorite Kubernetes feature and why?

Nicholas: Kubernetes secrets have always been a concern for me. The fact that they are stored in the cluster as plain text has been a challenge. Some organizations choose not to implement Kubernetes secrets and instead push credentials into the application itself rather than hosting them in Kubernetes secrets. It's not a perfect solution, but it's a necessary compromise.

Bart: Looking towards the next 10 years, what do you expect to happen?

Nicholas: I expect Kubernetes to manage more and more of the infrastructure. Like the Crossplane project and KubeFM, we are now managing virtual machines and infrastructure outside of the actual Kubernetes cluster. It's becoming a control plane and orchestrator for everything in your technology stack beyond just containers.

Bart: Now, on the subject of secrets and gaining security, two of our guests emphasize using multiple secret scanning tools to identify different vulnerabilities. They highlighted that each tool uses distinct methods, some of which are regex-based, while others are entropy-based, yielding varied results. What's your advice when it comes to securing and scanning for secrets, possibly using tools like External Secrets, and considering the broader context of cloud native technologies and Kubernetes security?

Nicholas: I haven't done much Kubernetes secrets scanning. I'm very opinionated that you should use something like External Secrets to manage those, but I don't have a strong opinion on scanning and securing them. That's fair.

Bart: Regarding secrets management and credentials, two of our guests described how they gained access to a Kubernetes cluster and moved laterally thanks to a credential that gave them push-pull permissions on an internal container registry. What's your advice for managing secrets and permissions in Kubernetes?

Nicholas: Policies are everywhere. It is essential to have a clear understanding of how to scope down your application as much as possible for the permissions you want to grant, following the principle of least privilege. Kubernetes makes this difficult, so it is crucial to find the right observability tools to understand the level of permission being granted to applications. Instead of putting the onus on developers to understand Kubernetes from the start, provide them with tools to help them comprehend the permissions they are granting.

Bart: What's next for you?

Nicholas: I am currently exploring the challenges of implementing technologies in government. It's an interesting space where the competition isn't as fierce, but the impact is much broader, and they have a broad mandate to serve the citizens of their country. I think it's a fascinating challenge to balance governance and innovation in a place where every choice made will affect possibly millions of citizens.

Bart: Great. With that in mind, thinking about scalability, which Kubernetes tools are you using currently to get that scalability in large organizations like governments?

Nicholas: Is it wrong to just say OpenShift? I have my bias now, being with Red Hat Canada. I really like the approach that they take, where OpenShift is not just Kubernetes, it's everything you need on top of Kubernetes, including monitoring, logging, and service mesh. This approach takes away the decision fatigue of choosing between different service meshes, which can take months to trial and decide upon, such as Istio and Linkerd. Instead, you're getting this entire platform that allows you to focus on scaling and deploying your applications, rather than deciding between Istio and Linkerd.

Bart: How can people get in touch with you?

Nicholas: LinkedIn is the best place to get in touch with me, to follow me. I post regularly about what I'm working on, the challenges that I'm seeing, and the solutions that we're seeing. I post about my conference experiences. That's the place where I like to connect with my community.