engineering
Engineering
How we run ALLSmartLLM: models, hardware, measurements and what went wrong along the way. From our own operations, with real numbers.
I fine-tuned a 4B model on 25 years of my own work
A model on a consumer GPU learned to build web components following my conventions. It worked, but the wrong measurement taught me more than the training.
Two models on one DGX Spark: 79 and 112 tok/s
A DGX Spark, two models with vLLM 0.28 and NVFP4. What went wrong, and how fast does it run? All the missteps and the solution are in the article.