01 the hardware · full parts list
every box, and what it's for.
the whole fleet sits on a desk and a shelf in augusta, georgia. no rack, no cloud, one power strip. this is the honest list: what each piece does, how it is wired, and what i would do differently.
gpu
ASRock Intel Arc Pro B6024 GB · ≈21 GB usable under Vulkan
the whole reason the fleet exists. roughly $500 for a 24 GB class card. serves the production 30B MoE at ~40 tok/s. compute-bound on the Vulkan backend, so quant choice matters more than file size.
pcie gen4 x4
over oculink
over oculink
dock
OCuLink eGPU dockGen4 x4 · powered by the 600W supply
the B60 stands on the dock next to the host. one cable to the EVO-X3's OCuLink port. four lanes are enough: inference is compute-bound, not bandwidth-bound, on this card.
external
power
Thermaltake Smart 600WATX supply, jumpered on
feeds the dock and the card. nothing else. plenty of headroom for a card that never pulls anywhere near its limit during inference.
on the dock
host
GMKtec EVO-X3AMD Ryzen AI Max+ · mini pc
runs llama.cpp, the memory brain, the agents and the dashboard. small, quiet, and the OCuLink port on the back is what makes a discrete card possible without a tower.
always on
storage
Aoostar WTR MAXNAS · models, snapshots, brain backups
every model file lives here and gets pulled to the host on demand. brain snapshots land here on a schedule. the drive leds you see in the photos are real activity.
on the shelf
standby
nuke-standbythe watchtower
a separate small box whose only job is to watch the fleet and heal it. if a service goes unhealthy it restarts it; if the host goes dark it tells me. never runs a model. never gets touched by an agent.
on the shelf
network
router + UPSon the same shelf as the NAS
the ups covers the shelf and the host long enough for a clean shutdown. outbound traffic from agents is gated on the router side too, not just in software.
on the shelf
desk
5120×1440 ultrawideshares the gpu when i am at the desk
worth stating because it changes the numbers: most benchmarks on this site were measured with the desktop sharing the card. rows marked idle were not.
display
how it's wired
[ desk ]
EVO-X3 host ──── oculink gen4 x4 ──── Arc Pro B60 on dock ◄── Thermaltake 600W
│ llama.cpp · neuralis · sentinel prime · aurora-x
│
[ shelf ]
├── Aoostar WTR MAX ─── models · snapshots · brain backups
├── nuke-standby ────── watches everything above, heals, alerts
├── router ─────────── outbound gate for agents
└── UPS ────────────── shelf + host, clean shutdown
[ software ] Vulkan · Mesa 26.0.8 · kernel 7.0.0-30 · llama.cpp b10588 / b10751
one model resident at a time · nothing leaves the house unless i say so
what i'd change
would do again
- the B60 over anything nvidia at the price. 24 GB is the gate for 30B-class models and nothing else gets there for $500.
- oculink instead of a tower. four lanes cost nothing measurable on this card and the desk stays quiet.
- a separate standby box. the watcher must not share fate with the thing it watches.
- one model at a time, enforced. co-loading spills to system ram and the driver never recovers gracefully.
would change
- size the model to ≈21 GB, not 24. the box-art number is not what vulkan exposes. i learned this the slow way.
- pick quants for the compute path. a pure Q4_K beats a mixed Q4_K_M by 3.5× here. i ran the stock quant for weeks first.
- benchmark idle and loaded separately from day one. the ultrawide sharing the gpu skews everything and i did not label it early enough.
- put the ups in before the nas, not after. one unclean shutdown was enough.