Plate IV · Engineering
The Polaris Works
The cards were fine. The software did not know what it was holding. Most of the speed came from fixing that.
Continuing
Every system on this site runs on AMD RX 580 cards, released in 2017 and sold by the pallet after the mining trade collapsed. Nothing modern targets them. The work of this house is the work that makes them useful.
Three findings worth the ink
The vendor compute stack drives one card per machine. Its atomic memory operations are not supported on this silicon, and the failure is silent: the machine reports six cards and uses one. The graphics driver has no such restriction. Everything here therefore runs on Vulkan, which drives all six.
The tuning path was dead because the detection was wrong. The inference server decided a card’s architecture by asking for a driver extension that this card does not publish. Every RX 580 was filed as unknown, so every tuned code path was skipped. Correcting the test took one attention kernel from three thousand and twenty-two milliseconds to one hundred and fifty-two.
Six cards in a machine were taking turns. Copies between cards were being issued in a way that made the machine wait for each one. Making those copies asynchronous took a prompt-processing benchmark from sixty-six to one hundred and fifty-nine tokens per second on the same hardware.
What we do with it
The fixes go back to the projects they came from. The rest becomes the plant: a build of the inference server that assumes this exact card, this exact driver and this exact processor, with the instruction sets these hosts do not have switched off.