> You are, maybe, just not the target audience here?
The difference between an open model running on a $100 computer and the output from GPT4 or Claude Sonnet is huge.
I use local and cloud models. The difference in productivity and accuracy between what I can run locally and what I can get for under $100 of API calls per month is huge once you get past basic playing around with chat. It’s not even close right now.
So I think actually you are not the target audience for what the parent comments are taking about. If you don’t need cutting edge performance then it’s fun to play with local, open, small models. If the goal is to actually use LLMs for productivity in one way or another, spending money on the cloud providers is a far better investment.
Exceptions of course for anything that is privacy-sensitive, but you’re still sacrificing quality by using local models. It’s not really up for debate that the large hosted models are better than what you’d get from running a 7B open model locally.
The difference between an open model running on a $100 computer and the output from GPT4 or Claude Sonnet is huge.
I use local and cloud models. The difference in productivity and accuracy between what I can run locally and what I can get for under $100 of API calls per month is huge once you get past basic playing around with chat. It’s not even close right now.
So I think actually you are not the target audience for what the parent comments are taking about. If you don’t need cutting edge performance then it’s fun to play with local, open, small models. If the goal is to actually use LLMs for productivity in one way or another, spending money on the cloud providers is a far better investment.
Exceptions of course for anything that is privacy-sensitive, but you’re still sacrificing quality by using local models. It’s not really up for debate that the large hosted models are better than what you’d get from running a 7B open model locally.