aka what i did to get from nothing to done.
note: these are designed to be primarily a re-install guide for myself (writing things down helps me memorize the knowledge), as such don't take any of this on blind faith - some areas are well tested and the docs are very robust, some items, less so). YMMV
| Architecture | NVIDIA GPU | Instance type | Instance name | Number of GPUs | GPU Memory (per GPU) | GPU Interconnect (NVLink / PCIe) | Thermal Design Power (TDP) from nvidia-smi |
Tensor Cores (mixed-precision) | Precision Support | CPU Type | Nitro based |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Ampere | A100 | P4 | p4d.24xlarge | 8 | 40 GB | NVLink gen 3 (600 GB/s) | 400W | Tensor Cores (Gen 3) | FP64, FP32, FP16, INT8, BF16, TF32 | Intel Xeon Scalable (Cascade Lake) | Yes |
| Ampere | A10G | G5 | g5.xlarge | 1 | 24 GB | NA ( |
