Expertise
Kubernetes engineer — since 2020
I have run containers on Kubernetes since 2020, model serving and inference workloads included.
- years
- 6
Me in the Kubernetes ecosystem
For me Kubernetes is the answer to how an application packaged with Docker stays up across several machines: where it runs, how many copies there are and what happens when one goes down are read from the declared state. The role side of this work is on the DevOps and infrastructure page.
Kubernetes sits where two roles meet. On the infrastructure side it is the layer that manages containers on top of Linux servers; on the AI systems side it is the platform where I deploy and scale model serving and inference workloads as containers. I cover that side separately on the AI Systems page.
What Kubernetes solves is scaling and restarting; how the application is packaged is a question for the Docker page.
What is the Kubernetes Engineer role?
Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services. It takes care of scaling and of restarting containers when they fail. A Kubernetes engineer is the person who sets up and operates this layer the containers run on.
What does the Kubernetes Engineer do?
I use Kubernetes as the orchestration layer where I run, scale and restart containers across machines. Model serving and inference workloads are part of that: I serve trained models behind an API with NVIDIA Dynamo-Triton. The application arrives as a [Docker](/expertise/docker/) image.
Technologies I pair it with
-
Kubernetes
The orchestration layer where I run, scale and restart containers across machines, model serving and inference workloads included.
-
Docker
Development environment and deployment unit.
-
Linux
The server operating system applications run on.
-
Terraform
For describing infrastructure in code and setting it up repeatably.
-
GitHub Actions
The CI/CD pipeline that defines test and deploy steps in the repository.
-
NVIDIA Dynamo-Triton
The model serving server, formerly NVIDIA Triton Inference Server, that I use to put trained models behind an API.
Questions in this area
Frequently asked
7 questions
-
How long have you worked with Kubernetes?
Since 2020 — 6 years.
-
What is Kubernetes?
Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services. It provides a framework to run distributed systems resiliently and takes care of scaling and failover.
-
What is Kubernetes used for?
According to the Kubernetes documentation it provides service discovery and load balancing, automated rollouts and rollbacks, resource placement, restarting failed containers and horizontal scaling. I use it to run, scale and restart containers across several machines.
-
What is the difference between Docker and Kubernetes?
Docker packages the application and its environment as a container; Kubernetes manages running, scaling and restarting those containers across several machines. The first is a packaging question, the second an orchestration question.
-
What workloads do you run on Kubernetes?
Applications packaged as containers, plus model serving and inference workloads. I use NVIDIA Dynamo-Triton to serve trained models behind an API.
-
What is the difference between Kubernetes and Terraform?
Kubernetes runs, scales and restarts containers; Terraform is for describing infrastructure in code and setting it up repeatably. They answer different questions and can be used together.
-
Where should you start learning Kubernetes?
Kubernetes manages containers, so you first need to understand what a container is. The Docker page and the Docker guide on the blog are a good first step for that reason.
Let us work together
If you have work in this ecosystem, tell me what you are building and we will talk about how to build it.