Bart Farrell: So first things first, who are you, what's your role and where do you work?
Koray Oksay: Hey Bart, my name is Koray, I work for Kubermatic. So maybe you know, but we have the Kubernetes management application. My role is mainly to deploy to customers and help their management of their environment. I also do trainings both for our applications and also for Kubernetes, Helm, etc. Cloud-native stuff. that's the main need.
Bart Farrell: Very good. So you're plenty busy with everything. So where does code quality break down first when changes span application code, Kubernetes configs, and CI pipelines in the same pull request?
Koray Oksay: Well, So when they are in the same pull request, it means that probably they are generated by the same person and if it's application then it's the developer and most of the time I see that developers are not that confident with Kubernetes or infra work, and in general, I would say that they know the basics, but they prefer that work be done by more experienced ops people. Those combined PRs often cause production configuration issues.
Bart Farrell: Why do YAML and Helm changes consistently escape the same scrutiny as application code? And what does that cost teams in production?
Koray Oksay: I mean, you know, Kubernetes manifest can be very long. And also if you have a Helm chart, like a big Helm chart with a lot of templating, it's really, sometimes it's really hard to follow what's happening based on the values.yaml you provide. So it is also quite hard to actually review them, understand the effect of it. Just adding one value in the values changes a lot of things, right? In the manifest that will be applied to production. So I think there should be some sort of pre-prod staging, whatever environment, but production-like environment where you test those kinds of configuration. Otherwise things would just slip through production and there will be some issues as well. So having a production-like environment is, I think, a key point here and test, of course, on this environment.
Bart Farrell: As AI writes more of the code going into Kubernetes deployments, what does good governance of that code actually look like?
Koray Oksay: I mean, if you look at it from the DevOps perspective, the governance should be on the CI, CD pipelines, right? I mean, at the very beginning of the stage, so that fast feedback, things would be covered earlier. But in the end, well, worst case scenario, there should be some sort of policy tool running on your clusters, like Kyverno or OPA or whatever you use to check the things that are actually good for production or good for your environment. AI helps a lot, for sure. But you have to check those kinds of Whatever it provides, you have to at least test it in advance. If not, make sure that there are some policies on your production environment that those changes are compliant with what you want to have on production.
Bart Farrell: Where are engineering teams getting AI code review right today? And where are they still flying blind?
Koray Oksay: I think things have changed in the last, I don't know, six months a lot. I mean, it used to be just ask the AI and it would provide you some information or review that kind of thing. But nowadays, AI is able to write an application from scratch in a couple of days, like a really huge applications or setups. So I think, well, using AI as a reviewer or getting some advice from it is still good and better than it used to be, obviously. But if you're making AI write the whole code, then reviewing is really hard. I mean, you simply can't, I mean, it's able to write 5,000 lines of code per day. So you can't review that code. It's not scalable for us to do it. But I think safest approach right now is to use AI as a reviewer and ask for small changes. So I think there are still hallucinations. We know that and it happens, but I think it's getting less and less.
Bart Farrell: What would it take for you to trust an AI recommendation on a production-bound infrastructure change? And what's the bar for that?
Koray Oksay: Well, I mean, I think it's not so ready for infrastructure yet. When I say I'm especially thinking about, for example, databases on cloud environments, like managed databases, et cetera, we've heard stories that it just removed the database, creates the new one, et cetera. That sort of stories are on LinkedIn nowadays. So I think especially for application, AI is capable of writing from scratch quite high-quality code. But for infrastructure, I think it's good to have it. It's still good to have it as a reviewer or get some recommendations, etc. But it will also change. I mean, obviously, maybe six months later, if someone is listening to this one, then they will say, hey, what are you talking about? I mean, we've been asking AI for everything. Probably we'll go there. But today, as of today, I think it's good as a teammate, reviewer, or peer developer, peer buddy, the day to trust it, I think, will come. I personally don't trust it at the moment for the whole infrastructure changes.
Bart Farrell: I've been doing podcast interviews for quite a while. It's the first time when someone says, when someone's listening to this in six months from now, I feel like we're in a time capsule. Someone at the British Museum. When Indiana Jones is trying to figure out where the arc of the curve is. All right. But, okay, but you did mention, you know, future forward, future looking, thinking about, you know, things that we can expect in the next six months. But, you know, Kubernetes is over a decade old now at this point and still accelerating. What does the next era look like for teams that are trying to maintain code quality at that scale?
Koray Oksay: Well, to be honest, I'm not very good at predicting the future. Also, I mean, it's getting really hard because things are changing quite fast and the whole AI stuff is incredible. I don't know. Honestly, I don't know. But things will be different from what we are doing today. And there is already a huge change. And I think what we are doing or how we operate today will be a bit different. Kubernetes, I think, will be here to stay, or at least a tool like Kubernetes, which does the whole management of containers, management of your applications, will be here, some tactics at least. Maybe a simpler one, because now people are starting to complain that it's too complex, it's too much, blah. I also see those kinds of feedback recently. I guess, I mean, it will be different. I just know that but I can't predict how easy or how different it will be in the future. But I think how we do things today is even different than what we were doing 10 years ago with Kubernetes. So I think AI will change this thing as well.
Bart Farrell: I think it's fair to say what you said, that no matter what happens, it'll be different than what would seem to be a button.
Koray Oksay: I don't know how different. I don't know in which direction, to be honest. But it will be different. I'm pretty sure about that.
Bart Farrell: Very safe bet. Now, more practically speaking, what are you focused on in building or solving next?
Koray Oksay: Scalability is always an issue. When you're working with Kubernetes, even if you're on the cloud, having scalable systems under load can still be based on what you are managing. The project runs on Kubernetes, and scaling it quickly is challenging. I think also on the cloud side, the hyperscaler side, there might be some improvement, or there should be some improvements as well to have more stable, more scalable environments. For example, nowadays we have an issue where we can't find VMs with GPUs in some regions, right? This is an issue. And adding more hardware with more GPUs is not the real solution. There could be some, I mean, there are some other ways to share them, et cetera. So I think, unfortunately, the resources are not unlimited. They are limited, and we should find a way to use them more efficiently.
Bart Farrell: And for people that want to get in touch with you, what's the best way to do that?
Koray Oksay: They can reach out to me on LinkedIn or by email, my work email or personal email. most of the time, on Twitter, X, and Bluesky. I try to answer all the questions. They can also reach out to me at KubeCon. I'll be there next week. I don't know when. I'll be in Amsterdam, so they can also find me there as well
