Bart Farrell: In this episode of KubeFM, I spoke with Federico Iezzi from Google Cloud about what it actually takes to serve open-source language models at very high throughput on Kubernetes. We used his work on reaching 1 million output tokens per second with Qwen 3.5, vLLM, GKE Autopilot, and NVIDIA B200 GPUs as a starting point. Federico explained why inference performance is often constrained by memory capacity and bandwidth rather than raw compute, how model architecture and KV cache configuration affect throughput, and why representative benchmarking matters more than producing an impressive headline number. We also discussed the operational side, GPU availability, scaling across nodes, load generation bottlenecks, and why a simple Kubernetes service using round-robin routing outperformed a more specialized inference routing layer for this particular workload. Listen to the full episode for a practical breakdown of how Kubernetes, vLLM, GPU architecture, and careful performance engineering come together in large-scale AI inference. Kubernetes requests and limits are easy to guess and hard to get right. Set them too high and you waste capacity. Set them too low and you risk throttling, OOM kills, or downtime. LearnKube has put together a free set of resources to help. A book, interactive calculators, and a cheat sheet for getting CPU and memory values right. You'll learn how to identify over-allocated containers and decide what is worth changing safely. Download them at learnkube.com/kubernetes-rightsizing. Now, let's get into the episode with Federico. You're tuned in to KubeFM. Welcome to KubeFM. What three emerging Kubernetes tools are you keeping an eye on?
Federico Iezzi: Hey, Bart. It's quite interesting. I feel like the problem of distributed serving is not entirely solved, and I'm very much into llm-d or NVIDIA Dynamo, for instance. And of course, not strictly Kubernetes related, but vLLM, it's like the thing that I check almost daily. So vLLM, I'm sure we'll talk about it later. It's an inference engine or inference server. It's where our models are running. Also important for really large serving platforms is LeaderWorkerSet. It's a custom CRD of Kubernetes. It's been around since at least a year and a half, but it's still fun and easy to set up and it integrates natively with the Ray frameworks. Another important one is Kueue, or I don't really know how to spell it, but it's a Kubernetes tool. This is so funny. Kubernetes was born out of ephemeral Pods and ephemeral resources, of course, besides traditional PVCs. And it's incredible that now there are so many interesting projects that are recognizing the extreme value of Kubernetes, but adding missing components. For instance, mapping more static, HPC-like resources in terms of placements, in terms of scheduling, in terms of allocation, in an environment that was originally not built with that thing in mind. Kueue is trying to solve that. Another recent technology that was GA in Kubernetes 1.35 is the In-Place Pod Resize. You're basically increasing CPU and memory on the fly on the same node. And again, as somebody who is in the industry since quite a few years, like OpenStack, in case somebody remembers it, that has been a struggle, a constant struggle, that I think it was never ever solved. Finally, Kubernetes has it. Last but not least, this is something that Google has recently announced. It's called Pod Snapshotting, leveraging the GPU memory allocation, and leveraging the cuda-checkpoint utility. It's a way to take a snapshot of the pod and effectively allows for a super-fast resume of those pods. This is important because the time it takes for an inference engine to spin up is usually long because when an inference engine starts, you have to carry out compiles, you need to load the model weights into memory. It takes a fair bit of time. This tool, especially with the CUDA checkpoint integration, aims to streamline this process and shorten the time that it takes. So it's less time wasted. But hope that's great for you.
Bart Farrell: It is. And Federico, for people who don't know you, can you tell us a little bit more about what you do and where you work?
Federico Iezzi: I'm a customer engineer at Google Cloud. I've been at Google for about five years. Customer engineer is a fancy name, but it's a technical pre-sales role. I'm based in Italy. I look after most of the digital native market in Italy. Customer engineers at Google sit at this pivotal role between the pre-sales aspects, POC, requirement gathering, economic aspects of every deal, but also looking after what happens once the customer has agreed to work with us, how they can effectively leverage our technology, how we can bring that thing to production. We follow both ends from the very beginning of the negotiations to production, supporting them all the way, connecting with PM, engineering and so on.
Bart Farrell: Good.
Federico Iezzi: Please try.
Bart Farrell: It sounds like you're quite busy. How did you first get into Cloud Native? Tell me about that.
Federico Iezzi: This is a funny story. My background is in telecommunications. I did network functions virtualization, or NFV, for a number of years, which was very much data center and cloud native. At the time, I was living in the Netherlands, in Amsterdam. Around 2018, I started seeing my teammates working with this thing called Kubernetes, which I heard about but never had any practical experience. At that time, the entire telecommunication industry was working on the new telco standard, 5G, although it was not official, it was not standardized, as we used to say. 5G became completely containerized and completely Kubernetes-driven. It started in a weird way from an industry that was not cloud-native. The 5G standard brought an influx of beautiful innovation. That changed me. My background before then was not as much cloud-native as it is today. That was quite a journey.
Bart Farrell: It's no secret that the Kubernetes and cloud-native ecosystem moves very quickly. How do you stay up to date with all the changes? What are the best resources for you to know about what's going on?
Federico Iezzi: My top-rated source is Hacker News. Hacker News doesn't just cover Kubernetes, but tech news. I'm following it through RSS feeds. I have one hour every day to catch up with the news. In the morning I catch up with what happened the day before in the US most of the time. It covers all the major tech news and most of those are cloud native. Some of those are Kubernetes-related, depending on the context. Hacker News is my go-to source.
Bart Farrell: Very good. And Federico, if you could go back in time and share one career tip with your younger self, what would it be?
Federico Iezzi: I'm always late with the latest trends. Although I read a lot, I spend time every day catching up with the news. Catching up with the news is different from playing with the latest technology. I wasn't late with Docker or virtualization or containerization, but because of the business side of things, I was late adopting and learning Kubernetes. I remember playing with Kubernetes the Hard Way, the famous GitHub repo that was instrumental in learning etcd and kube-scheduler and everything else. A tip for myself is not just dedicating time reading the news but dedicating some serious time into learning the latest technologies whenever possible.
Bart Farrell: As part of our monthly content discovery, we found an article that you wrote titled 1 Million Tokens Per Second: Qwen 3.5 27B on GKE with B200 GPUs. We want to take this further with a series of questions. Before we get into the million tokens per second result, let's establish the baseline. When someone serves an open source language model on Kubernetes, there is more involved than loading model files onto accelerators. What does a serving engine do? Give us some context.
Federico Iezzi: As I mentioned during the introduction, first of all, there is not one serving engine. There are multiple options in the industry, especially in the open source community. I tend personally to like a lot the vLLM inference engine or serving engine, but there are others like SGLang, for instance. The inference engine is the component where the model weights are being loaded. In consumer space, you have things like Ollama, which became super famous last year, and llama.cpp from Meta. Those are consumer-focused because on the enterprise side of things, usually you tend to reach things like vLLM or SGLang or others. Loading model weights, handling the hardware. GPUs or accelerators are there, but you need to load the model weight onto it. You need to compile the kernels, or specialized mathematical operations to actually perform matrix multiplication and all the other operations that an inference engine is about to perform, like multi-token prediction and other things. Managing the cache, or managing the KV cache, or in general, the transformer architecture for large language models is kind of a beast. It has so many moving parts or components and all of it is running on hardware on the GPU side. The API is exposed. All aspects of Transformer are offloaded onto the GPU. You need to program it. Model weights are so large. They can be tens of gigabytes. Some open-source models can even be hundreds of gigabytes or even shy of a terabyte worth of data. It needs to be downloaded somewhere. There are several approaches to solve this problem, but in the most basic and fundamental way, it's just kind of a git clone or pulling the data off of a Hugging Face. That also needs to happen. You need to retrieve this data. Inference engine is effectively the thing you talk to, and it's going to give you back an answer or tokens.
Bart Farrell: I think many teams out there assume higher throughput mostly comes from buying faster hardware. Your article tells a much more step-by-step story. What were you actually trying to improve when you started this experiment?
Federico Iezzi: I feel like what I was aiming at the very beginning was, okay, let's establish kind of a baseline of performance. Like, what can we get out of this hardware that I have available for this specific project? And also, that was, I believe, was not really published or reported upstream, but I also tested other hardware. So, ultimately, the hardware, strictly speaking, the hardware selection came down to, I have a bunch of requirements I have to meet from end customer. And working at Google Cloud, I don't have just the B200 Blackwell from NVIDIA. I have other GPUs available. So, what are the ones that, price performance-wise, can better answer the requirements? And so, and then it became just, okay, how can I, what can I get off of those resources, physical resources, and then seeing if we are too far off from the target or at a reasonable distance that tuning and optimization can get us there.
Bart Farrell: You chose the 27 billion parameter Qwen model for this benchmark. Model choice affects memory use, context length, quality, throughput, and cost. What made this model a good fit for the experiment?
Federico Iezzi: This answer has two parts. There is a business-driven part and a technical aspect.
Bart Farrell: Okay,
Federico Iezzi: The business-driven answer is that I originally didn't make the model choice. The model choice was customer, the customer was looking for, the customer actually arrived with this model, Qwen 3.5, and I believe, honestly, that they have made a great choice and we will get into the second part later on. But so the initial model selection, so we have internally established off of our team that the Qwen 3.5 dense model 27B is actually the right choice for our needs. But going into more technical aspects, so, Qwen 3.5 27B dense version is actually extremely good at following instructions, and that was instrumental for the customer use case. So that is more of the technical aspect. So the choice was effectively So this story that I wrote about needs to be plugged into a much bigger picture where actually there is a specific use case that has been solved, not just by me, but by a team of people. I was in charge of specific aspects, but ultimately it was like a business case. And again, Qwen 3.5 is particularly good at following instructions and the model is expected to receive lots of content as part of system prompts or part of the initial prompt. We needed a model good enough that wouldn't randomly hallucinate or anyways not follow instructions, right? So that was the reason for choosing it.
Bart Farrell: And the hardware in the article is the newer B200 generation. So instead of just thinking about accelerator count, you focused on memory capacity and memory bandwidth. What did that hardware change for inference serving?
Federico Iezzi: So in the In the inference world, there is a fundamental concept called the roofline model, basically. And this is actually something that I also realized not too long ago. I think there is a general understanding that nowadays we need faster or accelerators capable of processing more throughput. But in reality, what we need are accelerators that have the highest possible amount of memory and memory bandwidth, for one simple reason. So the inference phase of an LLM is divided into two phases. Excuse me, the inference of an LLM is divided into two phases, the prefill and the decode. The prefill part is the initial part when the prompt is being processed and it's merely compute bound. So the more parameters we have, the more matrix multiplication we need to perform, but that part is done once and at the beginning. The second part of the inference is called the decode phase, where effectively tokens are generated one after another in a sequential phase. And per each token, the inference engine needs to read the model weights. So in case of a dense model, the full model weights. In case of a MoE, Mixture of Experts model, just the amount of experts that needs to be read. So if we need to read 27 billion parameters at the BF16 that more or less translates into 27 GB worth of data, well, we need to read per each output token 27 GB of data. So this means that we need to have the fastest possible memory bandwidth in our system and that is instrumental. That also was the key reason for going for Blackwell B200 versus for instance Hopper H100 which are, don't get me wrong, amazing accelerator GPUs but they simply lacked enough memory bandwidth and to some extent also memory capacity to arrive with a reasonable number of GPUs to the target of 1 million token per second. Total output token per second.
Bart Farrell: And you ran this on, not surprisingly, Google Kubernetes Engine Autopilot. That changes the operational model compared with managing accelerator node pools yourself. What did Autopilot make easier and what still required planning?
Federico Iezzi: Okay, so Also here, kind of a twofold answer. So as a Googler, I have this incredible privilege to have access to Google Cloud, but I don't have access to an unlimited budget. So the same amount of money our customer pays, we are also charged too. I mean, they don't come off of my salary, thankfully, but we do have a budget that we have to respect. And this budget is quite tiny and very strict, especially when we have when we have GPUs or accelerators in the pictures. So for me, Autopilot is extremely good because it literally costs nothing. I don't spend, I don't throw away money just for a setup phase, for the configuration phase. Literally, it's a production-ready platform from the get-go and it takes minutes to set up and to configure. For the other aspects. What else needs to be taken into account during the planning phase? Well, it's GPU and accelerators in general, availability. Blackwell are extremely unavailable resources.
Bart Farrell: So you ran this on Google Kubernetes Engine Autopilot. That changes the operational model compared with managing accelerator node pools yourself. What did Autopilot make easier? and what still required planning?
Federico Iezzi: Okay, so I think also this one is going to be kind of a twofold answer. So as a Googler, I have this incredible privilege to have access to Google Cloud. I don't want to say with unlimited resources, but we have access to all Google Cloud products. But we also have a budget. We run on a very tight budget. So the incredible thing about Autopilot is that the entire platform end-to-end, it's fully managed. And I don't waste time, meaning also money, because I'm on a budget, right? For the setup, the configuration, driver, whatever. The platform is literally production-ready from the get-go, and it takes minutes. It's a simple API call or a simple CLI command, gcloud, a container create. So it's very simple. But what requires planning? Definitely what requires planning is accelerator obtainability. So it's not easy at all to obtain the latest generation of accelerators, like the B200. Those are in scarce amounts. And that was kind of a significant challenge also for me during this test for this customer, ensuring the capacity just for the test. At Google Cloud, we have, let's say, several ways how a VM or resource can be spin-up. That was not reported actually in the paper, but specifically for this work, I leverage on a concept called DWS Flex Start or Dynamic Workload Scheduler. It's basically a way to spin-up resources for a fixed amount of time, specifically seven days or up to seven days. and then automatically resources are reclaimed. And in that way, I was able to ensure capacity. But the capacity planning, as it's also called elsewhere, or in general, resource sustainability. It's kind of beyond technology, and it's extremely important, especially nowadays in the industry.
Bart Farrell: And your first major tuning decision was around how to spread work across the eight accelerators on a node. And for people that are out there coming from distributed training, splitting one model across all accelerators can feel natural. What happened when you tried that approach?
Federico Iezzi: Yes. So that is also, by the way, it's also called the tensor parallelism. And so one A4 node in Google Cloud has eight B200 GPUs. And of course, the B200s are connected with NVLink 5 going top of my head, I think 1.8 symmetric terabytes per second of bandwidth. But the problem is that this model, Qwen 3.5, it's so small that the inference engine was wasting so much time synchronizing the eight accelerators, the eight GPUs, and that resulted in kind of poor performance. I think going top of my head, I achieved something like 22,000 tokens per second, which was kind of almost nothing given the platform capabilities. And so first experience, not great, but I have to say.
Bart Farrell: After that, you switched to running independent copies of the model instead of splitting one copy across the node. That's a different mental model for serving. And how did that change the system's behavior?
Federico Iezzi: So first of all, Interconnect utilization or NVLink utilization dropped to zero because effectively running DP equal one, excuse me, DP equal eight, so one copy per accelerator, basically means that each accelerator takes care of its own model and vLLM will take care about the routing about requests per individual GPU. So interconnect utilization went to zero. And also, surprisingly, because the stream processors were not wasting time waiting for the request coming from other accelerators, also, let's say, the CUDA core utilization dropped to zero, basically. And not only that, but the total time per output token dropped. significantly latency, which is a median latency for all the output tokens. And throughput improved, I believe, by three folds from roughly 20K token per seconds to roughly 60K. So that was very surprising also for me because originally I was thinking, I mean, we have plenty enough bandwidth available, right? in the NVLink, but bandwidth is not important. The problem here is physics. It takes time to transmit data. And you can see the same concept in traditional computing besides HPC or AI inference with I/O wait. When CPU, it's literally burning CPU cycles waiting for the storage to provide the data. It's exactly the same concept. In this case, we don't have storage, it's an interconnect. And the data, it's not actually data, but it's synchronization between the various engines. But so the CUDA cores of each card were waiting for the others. And they were literally wasting time because it takes time to send and receive this data.
Bart Farrell: The biggest single node breakthrough came from changing how token generation used cache and prediction, without going too deep into every flag. What changed when you introduced those optimizations?
Federico Iezzi: so okay. So I think the biggest leverage besides DP, data parallelism, actually came from MTP, multi-token prediction, that gave, I think, a boost of 50% order of magnitude. So effectively, vLLM was able to generate one token ahead of time with the success rate of between 80 and 90 percent. So very effective. Beyond more than one, it was just kind of a coin flip. It was 50%. So that was wasting resources. The other breakthrough actually came from memory management. As I mentioned earlier, we need not just fast memory, but we need a lot of memory. The model is small enough that it's not consuming much of the HBM available. But the KV cache, yes. And vLLM is going to reserve per each request the amount of memory expected to be reserved. And I think originally was roughly 128K tokens. And that was a significant waste in resources. So that involved actually understanding with the customer, okay, tell me on average, how many tokens are you expecting to have in input and in output, and that allowed to tune the max sequence length, I believe to about 2K, 2048, which is again, it's very specific for our use case. But that allowed to shrink effectively how much KV cache per request we need and in terms that translates to have more memory available for more requests to be processed in parallel. And at the same time also quantization of the KV cache. I believe default is FP16 KV cache, and we went for FP8, which is basically lossless. Not myself, but somebody else in the team ran some evaluation benchmark and didn't see any drop in quality. So we established, it's a perfectly reasonable way, and just with a single flag, double the amount of available memory for KV cache.
Bart Farrell: Benchmarking model serving is easy to get wrong because different prompt and response lengths stress different parts of the system. How did you structure the benchmark and what should listeners check before comparing throughput numbers?
Federico Iezzi: So this is so interesting because unfortunately, inference benchmarking to some extent, and hopefully don't get me this in the wrong way, but inference benchmarking can be super misleading because there are so many parameters, so many aspects of the benchmark that can be tuned to have an incredible high number or an incredible result, but they might not be representative of your use case, of your situation. So again, I think understanding, having kind of a plan and a double check with the customer of what is your specific use case, what is the problem you're trying to solve, I feel is kind of instrumental. Also, specifically for this project, at the beginning we were working with a ShareGPT dataset, which is a dataset coming from, I believe originally, ChatGPT through a Chrome and Firefox extension that would allow user to share their input prompt and output answer. And we were using that to have like a variety mix of pretty much any kind of, all kinds of possible conversation to ChatGPT, but then it turns out that was not representative of what our customer was doing. So again, so benchmarking in LLM is not like running a SPEC CPU or other, or other very specific benchmark. It's about really understanding and having a good understanding: What are they trying to accomplish? And let's work on that. And let's try to reproduce that. Reproducibility and the representativity of the benchmark with a third party, with a customer.
Bart Farrell: And when you first scaled across multiple nodes, the numbers didn't make a lot of sense. More servers did not automatically mean more throughput. So how did you discover where the limit really was?
Federico Iezzi: So that was super confusing. I think doubling the number of GPUs or quadrupling it yielded a much lower result, and that made no sense whatsoever. So, before Google Cloud, when I was in telecommunications, I was actually a performance engineer. And as a performance engineer, you need to have lots of tools in your arsenal to understand what is going on. It could be perf, it could be Flame Graph, or it could be old school top command to understand, okay, let's see the resource utilization where the benchmark is running. Because I was personally under the impression that, hey, vLLM bench, it's multi-threaded, right? Must process, all those requests in parallel, it's not going to be single-threaded resource, right? Wrong. It was a single-threaded resource. So what I was originally doing was, okay, I now need to run a thousand requests in parallel. Let's go for a thousand requests and let's go for 2,000. Let's go for 8,000. But that was actually killing the benchmark and not stressing the nodes because the benchmark is, again, it's single node, just small caveat. This was all tested with vLLM 0.18 and 0.19. We are now at 0.22. So maybe in the past two, three months, the community specifically addressed this point. Actually, I'm very confident that they have. But back then, a few months ago, there was a vLLM bench, basically single-threaded, and top command spotted immediately the issue.
Bart Farrell: And once that load generation issue was fixed, the system scaled horizontally. From a Kubernetes architecture point of view, what made the node per replica design work so well?
Federico Iezzi: By the way, I didn't mention in my earlier answer, so the way of fixing the benchmark was simply running multiple copies in parallel. So I wrote a very simple shell wrapper that would spin up several vLLM bench and then aggregate the final results. So that was easy enough. Also with some kind of a warm-up point to make sure all models would be loaded in all the GPUs. Well, sometimes a very dumb, allow me to say, or a very simple approach, like a ClusterIP, it's working perfectly well, or round robin requests. And I have to admit, I believe we achieved 96 or 97% of scaling or parallel efficiency. Moving from 1 to 12 nodes, from 8 to 96 GPUs, again, 96 or 97 parallel efficiency with a simple round robin. And that was honestly quite impressive. Again, sometimes the kind of very old Unix concept, KISS, keep it simple. There's another S for something else. But sometimes the simple approach is the winning one.
Bart Farrell: And for those out there who are not familiar with the acronym KISS, it can be keep it short and sweet or keep it simple stupid. So you learn something new every day on KubeFM. Now, you compared the standard Kubernetes service approach with a specialized inference routing layer. Many people would expect the smarter routing layer to win. What did that comparison show?
Federico Iezzi: So that comparison actually showed that depending on the type of traffic that you are running, simple approach is actually massively better. The EPP, Endpoint Picker, which is a component, a much larger component called Kubernetes Inference Gateway, it's kind of a smart load balancer. Actually, not a kind of. It's a smart load balancer for the inference era, which consider a number of metrics like request queues, KV cache utilization. But in our situation, with the Qwen architecture and so on, basically, we were dropping requests. We were having something like 35% of throughput drop compared to a simple round-robin approach. But also here, on the Google Cloud reference architecture, which I have personally contributed, the inference gateway with the EPP, the Endpoint Picker, is on the reference architecture. So that would be our official suggestion. But of course, in such a fast-moving world like AI nowadays, it's super important actually to double check. I don't want to say double check everything, but definitely double check many aspects that you wouldn't otherwise. Because otherwise we would have been leaving 35% performance on the table, which is kind of a big hit, right?
Bart Farrell: In your article, you use an insurance document processing example to make the benchmark more concrete. And for teams out there that are evaluating self-hosted inference, what kind of workload makes these numbers meaningful rather than just impressive?
Federico Iezzi: Okay, so I kind of need to remove my hat from Google at this point and kind of speak as, I mean in my humble opinion, I think coding. So, and especially not just chatbot, but agentic coding. If you're leveraging an open source harness, like OpenCode, for instance, with some locally, with some open source model weights that you can locally deploy, when you're doing agentic coding, you are generating a lot of tokens. And I mean, I personally, through my various subscriptions or usage of Gemini, I can literally generate sometimes millions of tokens during the coding session or the vibe coding session. And that is me alone. And I'm not a software engineer. And I'm not a software engineer full time. I believe that is for people listening to us. So if you want to do vibe coding with your own hardware, own resources, I think, I mean, probably you're not going to generate a million tokens per second, but probably you might not be too far off. And especially in case you're leveraging like the strength of loops and goals when you have an agent working for you towards a certain goal or if you're doing sentiment analysis of your product on Twitter and Reddit and Discord this is also something that Anthropic recently disclosed with Boris, the creator of Claude Code, explaining how you always have something like, I think, 9,000 agents running in the background to check all sorts of things. And I mean, in that case, you really need to have beefy hardware. And those numbers do make sense.
Bart Farrell: Looking back at the full journey, this was not just one clever trick. It was a sequence of measurements of fixes across serving, memory, load generation, routing, and operations. What advice would you give? Sorry, I have to say that again because I said routing one after one be consistent. Because in Spain, we say rute. That's the problem. If learning how to say words incorrectly in your own language, then you're trying to say it. Anyway, so looking back at the full journey, this was not just some, you know, one clever trick. It was a sequence of measurements and fixes across serving, memory, load generation, routing, and operations. What advice would you give to a team trying to reproduce this kind of result on their own Kubernetes platform?
Federico Iezzi: Okay, so My personal take, and now putting back my Google hat, or in general, Hyperscaler hat. So when you're doing this kind of thing on your own Kubernetes and potentially on hardware, given how much expensive this hardware is, my, again, my personal view, but with Hyperscaler hat, I would strongly suggest to do some tests. They don't need to take months, they don't need to be extremely expensive, but do some tests in public cloud or not on-prem because you will be able to quickly reproduce your environment or the target environment without committing massive amounts of money. Then, of course, if you want to move on-prem for all sorts of reasons, control, safety, sovereignty, old school, private cloud, whatever, TCO, makes sense. But my personal suggestion is leverage resources on one of the cloud providers, whether one of the major ones or minor ones, to actually reproduce the environment. So this concept is very much kind of an engineering thing, but reproducibility, the environment, the solution, the prototype, the MVP that you're trying to test, try to make it as reproducible as possible with all sorts of tricks. So Infrastructure as Code is one of those, everything under Git. But not just that, make sure that there are no obvious mistakes. Double check with an agent, double check with Antigravity or Claude Code or Codex. I mean, leverage something to make sure that everything is done properly. Or even reach out to your representative at the public cloud how they would approach this. But that would be my kind of approach.
Bart Farrell: All right. And what's next for you? What can we expect next from you, Federico?
Federico Iezzi: Okay, so at the beginning, I kind of lied. You asked me about which ones are my three top Kubernetes projects I'm keeping an eye on. And I mentioned Dynamo and llm-d. And I'm very much looking at those, reading documentation, release notes, checking bugs or features, but it's not that simple to find actually some use cases for those. So definitely the next thing I'm planning to work on soon-ish is to work with either Dynamo or llm-d in a kind of a real context, and not just writing for the sake of writing, but more of kind of customer-driven. That is for sure my next step. And I believe that it could result in really interesting discoveries also and ways of improving even further inference at scale.
Bart Farrell: And if people want to get in touch with you, what's the best way to do that?
Federico Iezzi: Okay, so I check as much as I do no inbox policy, I also do no notification policy. So I kind of read and answer to everything. So either on LinkedIn, so look for my name and probably Bart, you're going to put my link in the description. Or on Medium directly. LinkedIn is slightly more business. Medium is way more technical. It's where I post, it's where I'm mostly active when I write things. So you can see all my work there.
Bart Farrell: Fantastic. Well, as you just mentioned, it definitely worked in our case, reaching out to you through LinkedIn. Medium is also a great place to do it. Most of all, thank you so much for sharing your experience with us and our audience. I'm sure people are going to gain a lot of knowledge from this and we'll probably have further questions about how they can implement some of the things that you mentioned when they're facing similar challenges in their work. I hope we can have you back on the podcast in the future. In the meantime, take care and be well. Cheers.
Federico Iezzi: Thank you so much.
