FlashAttention-4 Hits 1,605 TFLOPS on NVIDIA Blackwell GPUs

2 hours ago

NVIDIA's FlashAttention-4 achieves 71% hardware efficiency on Blackwell chips, delivering 3.6x speedup over FA2 for AI training workloads.