Kubernetes at CERN: Scaling Scientific Computing

Bart Farrell: Who are you, what's your role, and where do you work?

Ricardo Rocha: Hi, my name is Ricardo. I'm a computing engineer at CERN. I lead the cloud-native platforms as well as the machine learning platforms for all our campus and also the scientific workloads.

Bart Farrell: What are three emerging Kubernetes tools that you're keeping an eye on?

Ricardo Rocha: So for our use cases, we are mostly looking at our next generation detectors that are called High Luminosity Large Hadron Collider. And the tools that we need there are around machine learning. So we are following very closely the evolution of tools like Kubeflow, KServe, and also outside the CNCF MLflow. We are also looking at tools that ease the interactive access to these shared resources. And we are actually contributing to a tool called Container SSH, but we also support a lot of access and integration with other tools like notebooks, VS Code. And the one that is really critical for us as well is to improve our hybrid deployment. So tools like Kueue that allow us to improve the scheduling and the scheduling in cluster, but also the scheduling across multiple clusters and even external clusters. This is essential for what we are doing today.

Bart Farrell: Our guest, Marc Campora, on the subject of Kubernetes cost optimization, he thinks that with Kubernetes, it's quite easy to pay more than necessary because you pay for allocated or provisioned infrastructure, machines you start that are often underused. What strategies do you use to optimize Kubernetes costs?

Ricardo Rocha: In that respect, we are a little bit special because we run most of our workloads on-premises. So we have our data centers. So basically, we buy all the capacity in advance and then we use them. We just use external resources to burst from internal workloads. So in that respect, the way we optimize is to backfill holes. So anything that is idle, we try to put it at 100% or close to 100% 24-7. So if the workloads that are real or user workloads are not enough, we have a ton of backlog that we can use to backfill the holes in the whole infrastructure. If you're using the public cloud, though, it's a little bit more complicated, especially for GPUs. And I understand the question. If you're trying to get H100 GPUs these days or H200, you're probably getting something that is very close to having on premises where you reserve for 12 months. And I would say the strategy there should be exactly the same we use on premises and just to try to backfill whatever holes with additional workloads that you keep in a backlog.

Bart Farrell: Another guest of ours, Yue Yin, spoke about the project Gödel and Katalyst achieving 60% utilization. Yue shared, we think that the KubeScheduler could approach higher performance levels with further optimizations. However, there will be inherent trade-offs because the KubeScheduler prioritizes flexibility, extensibility, and portability. What's your perspective on this performance versus flexibility trade-off in Kubernetes?

Ricardo Rocha: That's a very complicated question, but I'll try to answer it. The Kubernetes Scheduler actually came a very long way. It's a very complex tool, and there's a ton of optimizations. One can spend an entire year looking at optimizing the different parts of the scheduler. It's also pluggable, so it allows us to use things like Kueue. So for our workloads, the optimizations we need are not the ones that Kubernetes was designed for, which is traditional IT services talking to each other with endpoints. What we need is batch workloads, interactive workloads, sharing resources. So all the traditional high-performance computing advanced features are what's important to us. So we've been working with Google and Red Hat and others to develop Kueue and help push Kueue. And this is to allow us to have features like post-scheduling or gang scheduling, fair share, and everything else we need to optimize our own workloads.

Bart Farrell: Kubernetes turned 10 years old last year. What can we expect in the next 10 years?

Ricardo Rocha: So, if the hype continues, I think we can expect this trend where the traditional high performance or scientific computing workloads have become everyone's problem. AI has been pushing, and especially Gen-AI has been pushing this quite a lot. What we also see from the hardware vendors is that instead of having the traditional commodity hardware, they tend to build these larger and larger machines. So Kubernetes will have to transition to be very good at maintaining not only very large clusters with many nodes that break constantly, but also these kinds of pet devices that are very similar to the mainframes we used to have. So it might be that it has to introduce functionality to manage time sharing on large devices and things that existed and were common in the 80s and the 90s. They might be coming back for the next couple of years.

Bart Farrell: What's next for you?

Ricardo Rocha: For me, this is it. It's what I mentioned, supporting our flagship project, which is the high luminosity Large Hadron Collider. Our challenge is that we have a limited budget that never changes. And when we generate more data, when our detectors generate more data, we don't get more money. So we have to find ways of dealing with the increase that is coming, which is 10 times more data than we have today with a similar budget. So we are going from one petabyte a second to 10 petabytes a second that we need to filter in near real time. And we have to find the technologies that will help us with this. And clearly AI will play a big role for that.

Bart Farrell: How can people get in touch with you?

Ricardo Rocha: So you can get in touch on LinkedIn. It's the best way. You can find me easily. Just send a connection and I'm happy to connect.