LLM Fine-Tuning, Compiling and Serving Optimization Benchmarks

The challenge

The objective was to build and benchmark a complete, production-aligned LLM pipeline for Qwen2.5-3B-Instruct — from distributed DPO fine-tuning through hand-optimized CUDA kernels, inference-engine selection, serving and Kubernetes deployment — and to do it with measured, reproducible numbers rather than vendor claims. The work was carried out on a Mac without CUDA and an AWS GPU quota that stayed at zero, so real validation was done on a rented NVIDIA A40; every figure below is from that hardware, with unverified parts named explicitly.

Approach

• Built a single DPO training-step implementation shared by Ray Train and Horovod, then trained Qwen2.5-3B-Instruct end-to-end on Anthropic HH-RLHF with both backends — loss decreased, no NaNs, and the comparison isolates the distributed layer.
• Wrote two hand-optimized CUDA kernels — a fused RMSNorm + per-token INT8 quantization kernel and a vLLM-style KV-cache paging kernel — and verified them against pure-PyTorch oracles across hidden sizes 2048/3072 and fp16/bf16/fp32.
• Benchmarked vLLM, TensorRT-LLM and ONNX Runtime GenAI on identical A40 generation workloads at batch sizes 1 and 8, capturing throughput, TTFT and GPU memory together so the throughput-vs-footprint tradeoff is visible.
• Validated serving end-to-end: real Triton Inference Server round-trips with dynamic batching and Prometheus metrics on CPU, plus real Qwen2.5-3B completions through the engines' native serving CLIs on the GPU.
• Deployed the orchestration layer on Kubernetes with plain HPA, KEDA driven by Triton's pending-request metric, and Istio weighted canary routing — all verified under load on a local cluster.
• Kept the repo reproducible: Terraform for AWS/EKS, versioned configs, raw CSVs and captured command output linked to each run. Multi-GPU scaling, Nsight profiling and GPU-scheduled Kubernetes were not verified because of the AWS quota and RunPod capability limits.

Outcome

vLLM was selected for serving, delivering 534.8 tok/s at batch size 8 — 7.2x over ONNX Runtime GenAI and ahead of TensorRT-LLM — with a 32.7 ms TTFT. The fused CUDA kernel achieved a measured 2.37–2.43x speedup with 15/15 correctness tests passing. Kubernetes autoscaling and Istio canary routing were proven under real load, and the entire pipeline is ready to deploy to a GPU cluster once cloud quota is available.