Hacker News new | past | comments | ask | show | jobs | submit login

I think regretting Mac upgrades is a real thing, at least for me. I got a 32G Mac mini in January to run local LLMs. While it does so beautifully, there are now smaller LLMs that run fine on my very old 8G M1 MacBook Pro, and these newer smaller models do almost all of what I want for NLP tasks, data transformation, RAG, etc. I feel like I wasted my money.



Small models retain much less of the knowledge they were trained on, especially when quantized.

One good use case for 32gb Mac is being able to run 8b models at full precision, something that is not possible with 8-16gb macs


Or better run quantized 14B or even 32B models...


Which ones in particular? I have an M2 air with 8GB, and doing some RAG development locally would be fantastic. I tried running Ollama with llama3.2 and it predictably bombed.


I would have thought that llama3.2:latest would run OK with 8G. Same for phi3.5:latest


You can sell it, get most of your money back.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: