AI Infrastructure
the compute fleet and the local inference and training stack that runs on it
Four compute nodes, each chosen for a distinct job, linked over a private tailnet: an AMD handheld (gfx1150) for portable work and on-device inference, a dual-RTX-3090 desktop holding the vector and graph stores behind a vLLM serving process, an NVIDIA GB10 (Grace Blackwell) desktop as the large-memory node, and a Dell PowerEdge VRTX chassis being provisioned as the always-on host — that provisioning is in progress, not finished.
On top of the fleet run two vendor tracks. The AMD track is ROCm-first PyTorch plus a controlled llama.cpp performance study of the handheld iGPU — a roofline, an eight-model ROCm-vs-Vulkan matrix, and a measured 2.43× speculative-decoding result. The study is written up in full, and the reproducible harness and raw results are public as gfx1150-bench.
The NVIDIA track is multi-GPU serving: vLLM on the dual 3090s behind a hand-written FastAPI router that decides model placement and concurrency, with the GB10 taking the models that don't fit elsewhere. That side runs on the fleet and has no public repo.
gfx1150-bench — inference tuning on a handheld iGPU
A controlled performance study on an AMD handheld iGPU: roofline, an 8-model ROCm-vs-Vulkan matrix, and a 2.43× speculative-decoding result.
gfx1150-bench
Reproducible llama.cpp benchmark harness and raw results for the AMD gfx1150 iGPU — a roofline, an 8-model ROCm-vs-Vulkan matrix, and a 2.43× speculative-decoding result.