NVIDIA A100: A GPU platform for AI and HPC
An overview of NVIDIA A100 workloads, AI training and high-performance computing use cases.
Key specifications
| Specification | Value |
|---|---|
| VRAM | 40GB / 80GB HBM2e |
| Memory bandwidth | Approximately 1.6–2.0 TB/s |
| CUDA cores | 6912 |
| Tensor cores | Third generation |
| FP32 | Approximately 19.5 TFLOPS |
| FP16 Tensor | Approximately 312 TFLOPS |
| FP64 | Approximately 9.7 TFLOPS |
| INT8 | Approximately 624 TOPS |
| TDP | 250W–400W |
| NVLink | Supported, 600 GB/s |
| MIG | Supported, up to 7 instances |
Technologies that distinguish A100
Multi-Instance GPU (MIG)
One GPU can be divided into up to seven smaller GPU instances, each running a separate workload. This is useful for cloud GPU services, Kubernetes GPU sharing and multi-tenant AI.
Third-generation Tensor cores
Tensor cores accelerate deep learning and Transformer models such as BERT and GPT. The source article describes training performance as approximately 6–10 times faster than V100.
High-bandwidth HBM2e memory
Bandwidth up to 2 TB/s helps reduce bottlenecks when training large models or processing large datasets.
NVLink and NVSwitch
High-speed links connect multiple GPUs, scaling from eight GPUs in a DGX system to hundreds of GPUs in a cluster.
Suggested workloads
- AI and machine learning: LLM training, including GPT and BERT, computer vision and recommendation systems.
- High-performance computing: physical simulation, molecular biology and quantitative finance.
- Data analytics: Spark, RAPIDS and big-data acceleration.
- Cloud GPU examples mentioned in the article: AWS p4d, Azure ND A100 and Google A2.
Comparison with other GPUs
| GPU | Positioning |
|---|---|
| A10 | General-purpose, lighter AI and graphics |
| A30 | Mid-range AI |
| A100 | High-end AI and HPC |
| H100 | A newer, more expensive and more powerful generation |
The article describes A100 as an established reference platform in many AI systems.
Advantages and limitations
- Strong AI performance and high memory bandwidth.
- MIG supports cloud resource sharing; NVLink supports multi-GPU scaling.
- High purchase cost and power consumption.
- Not well suited to small workloads.
Conclusion
Consider A100 for large-model training, including LLMs and computer vision, AI platforms, GPU clusters, HPC and large-scale data science. For light inference, graphics or VDI, the article suggests a smaller configuration or A10 instead.
Published ; updated