Prioritize GPUs with high CUDA core counts, substantial VRAM (e.g., NVIDIA's A100 or V100 Tensor Core GPUs), and support for hardware acceleration frameworks like NVIDIA's CUDA and cuDNN. Multi-GPU setups or GPU clusters can further boost performance, especially for large-scale training tasks
As of right now the only supplier that we can look at is Nvdia because of their CUDA core integration, thus limiting us to only a couple models:
NVIDIA RTX 3090
NVIDIA RTX 4090
- The 4090 series has a couple different variants on the market, but since we want to cluster them together, it might make more sense to look at liquid cooled instances.
NVIDIA RTX 4090 TI
- Looks like this might be delayed or may not even come out? I am not too sure.