Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
aschleck
on Nov 10, 2023
|
parent
|
context
|
favorite
| on:
Google Cloud TPU Multislice Training
1979 16 bit flops on an H100 is with sparsity. See footnote 2 on
https://www.nvidia.com/en-us/data-center/h100/
. You should be halving it for non-sparse flops.
YetAnotherNick
on Nov 11, 2023
[–]
GP is correct. With sparsity it is 3958. 1979 Tflop/s is without sparsity.
emu
on Nov 11, 2023
|
parent
[–]
No, it is not. That's the sparse fp8 flop number, but you need to ignore sparsity and compare bf16 flops not fp8 flops for the comparison the ancestor post is making.
Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
