Expertise
CUDA developer — the layer where a model meets the GPU, since 2024
Since 2024 I have run training and inference workloads on the GPU with CUDA, through PyTorch, TensorFlow and OpenAI Triton; I do not write CUDA C++ kernels.
- years
- 2
Me in the CUDA ecosystem
CUDA is not an application area for me; it is the platform layer through which a model reaches the GPU. A training or inference workload is written inside a framework, but how it runs on the GPU decides how much the system can carry. That is why I see CUDA as the infrastructure leg of AI systems engineering.
I reach CUDA through PyTorch and TensorFlow. Next to them are OpenAI Triton for inference optimization, NVIDIA Dynamo-Triton for serving models and Kubernetes for scaling.
CUDA is, for me, the name of the platform underneath the work.
What is the CUDA Developer role?
CUDA is NVIDIA's platform for accelerated computing and the foundation for GPU computing. A CUDA developer is the person who develops code that runs on the GPU with CUDA. I do not use CUDA by writing CUDA C++ kernels directly; I use it through PyTorch, TensorFlow and OpenAI Triton.
What does the CUDA Developer do?
I run training and inference workloads on the GPU with CUDA through PyTorch and TensorFlow. For inference optimization I write custom GPU kernels with OpenAI Triton, serve models with NVIDIA Dynamo-Triton and scale these workloads as containers on Kubernetes.
Technologies I pair it with
-
CUDA
The NVIDIA platform I run training and inference workloads on the GPU with.
-
PyTorch
The deep learning framework I use for model training, fine-tuning and adaptation with LoRA/PEFT.
-
TensorFlow
The deep learning framework I use in my model-training work.
-
OpenAI Triton
The language and compiler I use to write custom GPU kernels for inference optimization.
-
NVIDIA Dynamo-Triton
The model serving server, formerly NVIDIA Triton Inference Server, that I use to put trained models behind an API.
-
Kubernetes
The orchestration platform I use to deploy and scale model serving and inference workloads as containers.
Frequently asked
6 questions
-
How long have you worked with CUDA?
Since 2024, which is 2 years.
-
What does a CUDA developer do?
They develop code that runs on the GPU with CUDA. My scope is narrower: I run training and inference workloads on the GPU with CUDA through PyTorch, TensorFlow and OpenAI Triton.
-
How are CUDA, PyTorch and TensorFlow related?
PyTorch and TensorFlow are the frameworks I define and train models in; CUDA is the platform that runs those workloads on NVIDIA GPUs. For the framework side see the [PyTorch](/expertise/pytorch/) and [TensorFlow](/expertise/tensorflow/) pages.
-
Do you write CUDA C++ kernels?
No. I use CUDA through PyTorch and TensorFlow; the custom GPU kernels that inference optimization needs, I write with OpenAI Triton.
-
Whose platform is CUDA?
It is NVIDIA's own platform. It is the one I use, and I make no claim about any other GPU platform.
-
How do you scale GPU workloads?
I deploy model serving and inference workloads as containers and scale them with Kubernetes.
Let us work together
If you have work in this ecosystem, tell me what you are building and we will talk about how to build it.