Trust data on Kubernetes and building operators like a human
Discover the challenges and advancements in running data on Kubernetes.
In this interview, Jonathan Battiato discusses:
The complexities of running data on Kubernetes due to trust and storage issues and how solutions like Longhorn and Portworx address these challenges.
The operator's role in managing PostgreSQL clusters in Kubernetes using extended controllers and CRDs.
Relevant links
Transcription
Bart: Who are you? What's your role? And who do you work for?
Jonathan: Hello, I'm Jonathan Battiato, and I work for EDB as a support engineer and quality assistant for the Cloud Native PG software and related projects.
Bart: What are three emerging Kubernetes tools that you are keeping an eye on?
Jonathan: I'm most interested in the storage part, so I'm looking for a solution that works for storage, some kind of solution. That takes into account reliability and performance for network storage. I'm looking at Portworx, Longhorn, and all the stuff around the hardware. There is another company that implements NVME over TCP, which I'm interested in. I don't remember the name. It's like Lightbits.
Bart: One of our guests, Steven Sklar, shared that you can and should run a database on Kubernetes. The tooling and practices have matured since Kubernetes began, and you should run stateful applications there. What's your experience and advice with running stateful applications in Kubernetes?
Jonathan: As part of EDB, I usually run data on Kubernetes, and we work towards allowing everyone to run data on Kubernetes. I personally find it a bit challenging right now because of the current state of research around data and storage. Since we are working together to advance the data and Kubernetes community, I think we are moving in the right direction to enable everyone to run data on Kubernetes. The primary challenge is trust. Trust is the first obstacle to running data on Kubernetes. Then there are other concerns like reliability, availability, and locality. These are all elements that can be addressed by any storage driver, for example, Longhorn or Portworx.
Bart: Steven shared some simple but effective advice on building Kubernetes operators. Keep it simple and use multiple CRDs. Do you have any advice on operators?
Jonathan: The operator should be taught as a human operator that acts on the state of the cluster to maintain it. For us, it's for PostgreSQL, for example. When we build a PostgreSQL operator, we think about what a DBA should do when a failure happens. We expand the controller of Kubernetes through the actual control of the operator and through the CRDs, using different resources to allow Kubernetes to learn about Postgres clusters.
Bart: What's next for you?
Jonathan: I think I will try going to benchmarking storage, and of course, continue working on Cloud Native PG.
Bart: What kind of contributions have you made to Cloud Native PG?
Jonathan: I'm mostly testing it and finding bugs before customers and users do. My contribution is to allow everyone to have production-ready software.
