How many t/s would you expect? I think I feel perfectly fine when its over 50. A... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		mrtksn 5 months ago \| parent \| context \| favorite \| on: Apple M3 Ultra How many t/s would you expect? I think I feel perfectly fine when its over 50. Also, people figured a way to run these things in parallel easily. The device is pretty small, I think for someone who wouldn't mind the price tag stacking 2-3 of those wouldn't be that bad.

yk 5 months ago | [–]

I think I've seen 800 GB/s memory bandwidth, so a q4 quant of a 400 B model should be 4 t/s if memory bound.

behnamoh 5 months ago | [–]

I know you’re referring to the exolabs app, but the t/s is really not that good. it uses thunderbolt instead of NVlink.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact