Kubernetes evolution: automation, cost optimization and the AI-driven future

Bart: I'm Guy Baron. I work for ScaleOps, and my role is [specific role to be clarified].

Guy: Hi, I'm Guy. I'm the co-founder and CTO of ScaleOps.

Bart: What are three emerging Kubernetes tools that you are keeping an eye on?

Guy: That's a good question. If I need to choose only three technologies I'm very interested in, I'm keeping an eye on their direction. Obviously, Karpenter is being adopted by cloud providers. AWS is using their auto mode, and it's interesting to see where they'll take it.

Karpenter and how it's being used by cloud vendors is definitely something I'm tracking. Cilium is always interesting, and eBPF technology brings remarkable capabilities to the platform for observability and how it's being used in microservices and general architecture. Seeing the adoption of Cilium and eBPF across the board is something I keep track of.

Lastly, I'm always interested in multi-cloud, multi-cluster architectures and deployments. Karmada is an interesting tool for multi-cluster and multi-cloud scheduling. Those are my current favorites.

Bart: Our podcast guest, Alexandre, wrote an article about over-provisioning. What strategies have you found effective in controlling over-provisioning in large Kubernetes clusters?

Guy: In ScaleOps, we've seen thousands of clusters in production, particularly large clusters. I think the real answer to over-provisioning and strategy is automation. Once your cluster hits a certain scale, whether by resources or workload amount, processes and tooling tend to break. The best strategy we've found is for DevOps, platform, and SRE teams to invest in automation—specifically automation of provisioning, resource allocation, and resource requests. I think this is the best strategy to deal with over-provisioning at scale.

Bart: Our guest said that he's been looking forward to in-place pod resizing for years. Karpenter turns 1.0. It's an exciting time for scaling and resource optimization in Kubernetes. Where do you see this field evolving in the future?

Guy: We've been waiting for in-place updates for years, and it's really exciting to see the progress the team has made over the past few months. With Karpenter and the trajectory of the software industry creating more complex software, fueled by AI, I think the amount of software artifacts will increase exponentially.

There will be a significant focus on automation and scaling across every dimension of the cluster and workload—be it resources, number of replicas, pod level, workload level, node level, or even multi-cluster level. Auto-scaling is going to evolve tremendously over the next few years, driven by AI advancements and the increasing volume of deployed services.

The next steps for scaling are multi-dimensional. Organizations and teams will need to up their game and focus on auto-scaling across every aspect of their workloads.

Bart: Kensei suggested that developers should ideally handle pod-level optimization, such as setting requests and limits. How can organizations bridge the knowledge gap for developers who aren't Kubernetes experts?

Guy: Historically, as software evolves, the level of abstraction gets higher, and the details and inner workings of software become encapsulated into infrastructure. The same would apply here. There's an emergence now of infrastructure and platforms that hide the complexity of managing resources and the internal workings of Kubernetes. Workload settings and various aspects are being automated for developers, exposed at a higher, more abstract level.

The way forward is encapsulating more concerns in platforms that automate different aspects of underlying infrastructure while exposing developers to higher-level abstractions regarding their workloads, cluster dynamics, and traffic patterns. This approach bridges the existing gap between infrastructure complexity and developer experience.

Bart: Miguel Bernabeu-Diaz shared that Adevinta migrated their Kubernetes cluster to support ARM instances and workloads because they are more cost-effective. Do you have any practical advice on reducing your Kubernetes costs?

Guy: Reducing Kubernetes costs seems like a simple problem, but you need to approach it from a multi-dimensional perspective. You need to optimize resource allocations for workloads, selecting proper node types, CPU architectures, and deal with horizontal scaling. The best advice is to approach it from a holistic view and try to optimize all different facets of the problem to gain significant cost savings.

It seems simple, but if you don't address all dimensions of the problem, you'll fall short. My advice is to tackle cost optimization at each layer—from workload to vertical scaling, horizontal scaling, and node selections.

Bart: Kubernetes turned 10 years old last year. What should we expect in the next 10 years?

Guy: Software is changing. The rise of LLMs and RAG architectures will change the way software is going to be built. I expect much more software to be created over the next years. The growth in software architectures is not growing linearly, but exponentially. We'll see much more adoption of Kubernetes, containers, and complex architectures.

Kubernetes turns 10, but due to the rise of AI driving new software creation models, I believe it's only the beginning. Kubernetes is the operating system of the future, and I expect it to grow exponentially over the next few years, driven by AI. That's my personal opinion.

Bart: What's next for ScaleOps?

Guy: I love the progress we're making at ScaleOps helping our customers and users work better with Kubernetes, saving costs and increasing performance. I'm excited to see the future of ScaleOps and working with our users and customers to simplify cost savings and Kubernetes adoption. That's my future, and I'm looking forward to interacting with anyone.

Bart: How can people get in touch with Guy Baron?

Guy: Feel free to reach out to Guy Baron on LinkedIn, ScaleOps, Twitter (@rhinof), and other social media platforms. Just say hi.