Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's going to be faster no matter what. My M3 MAX prints tokens faster than I can read for the new MoE models. It's the prompt processing that kills it when the context grows beyond a threshold which is easy to do in the modern agentic loops.


If your computer was faster at it, you could run more capable models at the same token rate.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: