Beyond Kubernetes: Serverless Execution Models for Variable Workloads

Bart: In this episode of KubeFM, we're joined by Marc Campora, a systems consultant with experience ranging from military-grade C++ development to high-throughput Java platforms and cloud-native architecture at scale. In this episode, Marc shares insights into platform decisions based on cost models, infrastructure utilization, cold-start behavior, and operational overhead.

We explore how Kubernetes scheduling, cluster topology, multi-AZ deployment, and resource bin packing introduce non-trivial complexity, particularly for services with variable load profiles. Marc contrasts this with serverless execution via AWS Lambda, covering invocation patterns, warm start strategies, runtime limitations like single-threaded execution, and no in-function caching, and the effect of concurrency quotas at scale.

If you're evaluating execution models, revisiting platform choices, or debugging infrastructure cost creep, this conversation offers practical clarity that you will find insightful.

This episode of KubeFM is brought to you by Learnk8s. Learnk8s has been helping engineers level up their Kubernetes skills since 2017. This is done by offering courses that are 60% practical and 40% theoretical, given to individuals and groups, both online and in person. Students have access to course materials for life, and you can find more information at LearnK8s.io.

Now, let's get into the episode. Marc, welcome to KubeFM. What are three emerging Kubernetes tools that you are keeping an eye on?

Marc: I'm not a Kubernetes specialist, but I keep an eye on all optimization techniques. The team we are working with at the moment started to deploy Karpenter, I'm looking at this as a way to optimize infrastructure usage. I'm also looking at what AWS is doing with Fargate to run containers on microVMs.

Bart: Fantastic. For people who don't know you, Marc, can you tell us about what you do and your work?

Marc: I'm an independent consultant in technology. I worked many years for a big travel tech company, processing millions of transactions. Then I created my own startup with a couple of colleagues. At the moment, I'm working for a company developing a search engine for academic and medical content articles. I'm helping them make sense of their architecture and roadmap.

Bart: How did you get into Cloud Native?

Marc: How did I get into Cloud Native? The travel tech company I mentioned decided to modernize their architecture six or seven years ago. They wanted to move to the cloud progressively because we had a big data center and private infrastructure. They didn't want to rewrite everything. The architecture was already a service-oriented architecture, so we picked Kubernetes as a way to repackage or reencapsulate all the services. We picked Azure as a public cloud provider. That's when I discovered the public cloud and cloud-native initiatives.

Bart: What were you before cloud-native?

Marc: I started as a C++ developer for military-grade projects. Then in the early 2000s, with the boom of the internet, I became a Java architect. And all that without the help of Stack Overflow or GitHub Copilot. So I'll let you imagine that was another world.

Bart: In the Kubernetes and cloud-native ecosystem, things move very quickly. How do you stay up to date with all the changes that are going on?

Marc: I read a lot of blogs, newsletters, and everything. I don't have a favorite one. At the moment, I'm reading a lot of Medium articles. Your video and blog are also part of the mix. But I don't have any true favorites. The real thing I'm trying to do is when I discover something interesting, I spend time playing with it, writing a smaller prototype. Because I need it to truly understand how things are working.

Bart: If you could go back in time and share a career tip with your younger self, what would it be?

Marc: Keep learning, do it even more because that's a very rewarding part of the process or journey. Keep experimenting, trying new things, because this part is really exciting. That's what makes things interesting, and you grow by doing that.

Bart: As part of our monthly content discovery, we found an article you wrote titled "Lambda or Containers" (Note: No specific link was provided for this article). Kubernetes has been a popular choice for container orchestration, but we're seeing some organizations reconsidering their approach. What challenges are these organizations encountering with Kubernetes that are prompting this re-evaluation?

Marc: The company I'm working for at the moment faces difficulties with Kubernetes. That's why I started working with them. I began looking at their deployment because they were experiencing multiple outages and stability issues. They started to question if Kubernetes was a valid choice and whether they should move everything to AWS Lambdas or functions.

Let's start with the complexity. When you run Kubernetes deployments, you have many non-trivial decisions to make. You need to decide: Am I going to run one cluster or multiple clusters? Which instance types will I pick at infrastructure level? How many machines or instances do I need? Then you need to determine how to spread these machines across multiple availability zones for high availability.

These decisions are complex, and you can easily make mistakes. To make these decisions, you need specialists. The learning curve is quite steep, so you must find experienced professionals. The operational overhead is significant.

Regarding cost, it's quite easy to pay more than necessary because you pay for allocated or provisioned infrastructure—machines you start that are often underused. It's common to have pods and containers requesting a large minimum set of instances that remain idle most of the time. You encounter a bin packing issue on instances, with spare capacity always left unused. This is multiplied when deploying across multiple availability zones, resulting in a significant bill.

Even with dynamic resource allocation, your traffic evolves throughout the day, making it difficult to find the right infrastructure level depending on the time of day, week, or season. These complexity and cost factors led people to question whether they should move to Lambda.

Bart: Now you mentioned Lambda functions as an alternative. For those out there who might not be familiar, could you explain a little bit about how the Lambda model differs from the container-based approach that's using Kubernetes?

Marc: With Lambda, you don't have to take care of the underlying platform. You write your code, package it, and deploy it. The provider, AWS in the case of Lambda, takes care of everything. They provision the proper infrastructure, scale it depending on the demand and incoming requests, and spread this infrastructure across availability zones to ensure reliability. All this is essentially free when you pay for it, but you don't have to worry about the details. It's a very efficient and productive model from both development and operational standpoints.

As Adrian Cockcroft once said, you can deploy a complete application with AWS Lambda in the time it takes to decide how to deploy Kubernetes. This illustrates AWS's strength: with Lambdas, you can move quickly while AWS handles many of the non-trivial infrastructure decisions.

Bart: Now, an element that's very important here is cost, as it is a driving factor in many of these decisions. How does the pricing model for AWS Lambda compare to running containers in Kubernetes, particularly for different usage patterns?

Marc: With containers, you have to provision infrastructure and machines to run your containers. You pay for the time these machines are provisioned. With Lambda, it's a pay-per-use model: you pay only when the function is invoked, typically a few cents per call. If the function is not called, you pay nothing, which can be very cost-effective in certain situations.

When I looked at my customer's deployment, they had multiple systems: test and production. On the test system, which had over 500 microservices, the traffic was low—no traffic during nights and weekends, and minimal traffic during the day. Comparing their Kubernetes costs to potential Lambda costs, the savings were significant. They could potentially reduce costs by three times on the test system.

In production, the picture was more nuanced. About 80% of the 500 services were mostly idle, which would have been more cost-effective using Lambda. The remaining 20% were processing traffic consistently throughout the day. Moving these to Lambda would have been more costly. Ultimately, the total cost comparison balanced out.

The conclusion is that it depends. If you have consistent and high traffic on a function, Lambda can be more expensive than containers. Conversely, if your service has infrequent and variable traffic, Lambda becomes more cost-effective.

Bart: That's interesting, because it seems that the cost benefits can vary depending on the use case. How does AWS Lambda compare to containers in scenarios with more consistent, high-volume traffic?

Marc: For consistent traffic, containers are more effective. There are other scenarios where AWS Lambda is not necessarily appropriate. If you have a need for high computation, requiring significant GPU or CPU power, it's probably more effective to use containers and control the underlying instances more precisely.

With Lambda, the execution environment starts on demand when processing a request. This leads to what they call a "cold start" issue. When launching the environment, you have to wait a bit, and if your application or function is complex with many embedded libraries, you might experience variable latency and long response times.

If you need to create applications that require quick and constant response times, Lambda is probably not the best solution.

Bart: Lambda functions operate within AWS's shared infrastructure, which likely comes with some constraints. What limitations or quotas should developers be aware of when using Lambda at scale?

Marc: By default, Lambdas run and scale inside a region, which is a unit of infrastructure for AWS. You are using all the available capacity in this region. However, as many customers are working in the same region, AWS ensures a fair distribution of capacity and protects itself from overload.

The first quota you'll encounter is that, by default, your account cannot launch more than 1,000 execution environments for a given Lambda. Once you reach this limit, requests will be queued, waiting for an available execution environment to process them. This quota is negotiable, and you can ask AWS to extend it.

The second limit is non-negotiable. Even if you request more, you have a maximum of 2,000 execution environments for a given Lambda, and you cannot allocate more than 1,000 during a 10-second window. This is a way to protect the region. If you have a surge of traffic, AWS will let you allocate 1,000 instances immediately, but then you must wait 10 seconds to allocate more.

If you have a highly trafficked application, you might face exceptions and timeouts because AWS will throttle and reject or queue requests to protect its infrastructure.

Bart: It's a long one, but a good one. For teams considering migrating existing applications to AWS Lambda, what key factors should they keep in mind during this process?

Marc: The second question from my customers was: Can we migrate existing microservices to Lambda? After careful analysis, it's a non-trivial project with significant risks. While it's relatively easy to adapt services to appear as a Lambda and redeploy them, the main challenge is changing the execution runtime model.

For instance, Lambdas can go up and down quickly, which means:

• You cannot use reliable caching inside your function

• Multi-threading is not feasible

• Lambdas are designed to process one request at a time

Additionally, there's the cold start issue, which is particularly significant with Java services. The JVM is slow to start, and loading numerous libraries can result in unacceptable cold start times. AWS has released features to optimize this aspect, but the bottom line remains: migrating existing microservices to functions requires substantial effort. You need to test, transform, and potentially rewrite the services entirely.

Bart: In other conversations, we've given a lot of focus primarily to AWS Lambda, but there are other serverless function services available. How do offerings like Cloudflare Workers, Val.town, or Azure Functions compare to AWS Lambda?

Marc: Azure Functions and the equivalent on Google Cloud Functions are very similar to AWS Lambda, one-to-one. When I looked quickly at Cloudflare Workers, I think they compare more to CloudFront functions. Cloudflare is a CDN, a globally distributed network, and what they offer with workers is the capability to run your function very closely to your end users to reduce latency. That's a great idea—you can push part of your logic close to your user so they don't have to cross the world to connect to your servers. However, there is a trick: you cannot really replicate your backend application all around the world. So it's only part of your logic that you can push.

Bart: Some have suggested running AWS Lambda on Kubernetes as a way to get the best of both worlds. What are your thoughts on this hybrid approach of combining Lambda with existing containerized services?

Marc: It's an interesting approach for the bin packing issue and idle capacity left on instances, as it will be easier to use this unused capacity. But in my opinion, you missed the point if you do that, because you still have to take care of Kubernetes. The complexity aspect requires you to deploy, resize, and operate it correctly.

My opinion is that it's probably better to blend or mix both architectures. I have a Kubernetes deployment with a few services requiring containers, and Lambdas using the native AWS or Azure environment to operate them. The only thing you need is a good level of abstraction, so you don't actually know if it's running on Lambda, containers, or Kubernetes. This is where service bus or service mesh becomes agnostic.

Bart: To conclude our discussion, Marc, could you summarize the main advantages of AWS Lambda and identify the scenarios where it might be the most suitable choice for development teams?

Marc: If you start a greenfield project and are not sure about market fit, and don't know how much volume you will process, consider using AWS Lambda. It's a very productive development and operational environment that is cost-effective for this type of project because you will process only a small amount of traffic initially. If you succeed and scale, you will only need to rewrite 10 or 20% of your functions to optimize costs. So if you start a project, go for it.

Bart: What's next for you, Marc?

Marc: At the moment, I'm looking at data platforms, the Lakehouse buzzword. I'm helping my customer modernize its data platform. We are looking at ways to create and share data assets using Open Table standards.

Bart: What's the best way for people to get in touch with you?

Marc: I'm available on LinkedIn. That's my main communication platform. I usually respond quickly to people who contact me, so do not hesitate.

Bart: Good. Marc, thank you so much for joining us today. I hope our paths cross in the future. Take care.

Marc: Thank you, goodbye.