APU for LLM Inference?

[AstonJ]

@AndyL

Example APU machines include the Framework Desktop and the Mac M3 Ultra.

What about a cluster of M4 Minis? (or Mac Studios)

@AstonJ

This is really cool:

https://digialps.com/deepseek-v3-on-m4-mac-blazing-fast-inference-on-apple-silicon/

They’re running the full 671 billion parameter model on a cluster of 8 Mac Minis, each with 64GB of RAM.