Accelerator catalog
Capacity is only one axis.
Memory bandwidth, usable capacity, software maturity, power, supported runtimes, topology, and system integration all affect a recommendation.
NVIDIAexcellent stack
NVIDIA RTX A2000 12 GB
llama.cpp · vllm
Memory / bandwidth12 GB288 GB/s
Price observation$250+used-estimate
NVIDIAexcellent stack
NVIDIA GeForce RTX 3060 12 GB
llama.cpp · vllm · sglang
Memory / bandwidth12 GB360 GB/s
Price observation$180+used-estimate
NVIDIAexcellent stack
NVIDIA GeForce RTX 5060 Ti 16 GB
llama.cpp · vllm · sglang
Memory / bandwidth16 GB448 GB/s
Price observation$430+range
NVIDIAexcellent stack
NVIDIA GeForce RTX 3090 24 GB
llama.cpp · vllm · sglang
Memory / bandwidth24 GB936 GB/s
Price observation$650+used-estimate
AMDdeveloping stack
AMD Radeon AI PRO R9700 32 GB
llama.cpp · vllm
Memory / bandwidth32 GB640 GB/s
Price observation$1,250+current-retail
NVIDIAexcellent stack
NVIDIA GeForce RTX 5090 32 GB
llama.cpp · vllm · sglang
Memory / bandwidth32 GB1792 GB/s
Price observation$3,500+range
Applegood stack
Apple M3 Ultra 96 GB unified memory
llama.cpp · mlx-lm
Memory / bandwidth96 GB819 GB/s
Price observation$3,999+range
NVIDIAexcellent stack
NVIDIA RTX PRO 6000 Blackwell 96 GB
llama.cpp · vllm · sglang
Memory / bandwidth96 GB1792 GB/s
Price observation$13,250+current-retail