So all products must be available from one specific, primarily US focused, retailer of consumer-grade PC parts? I'm not sure I understand what your criticism is here.
That the author of the post wants such parts to be available and affordable as hardware you can put under your desk instead of something you only get to rent at Googles fickle will until they cancel it too.
Using https://github.com/kvcache-ai/ktransformers/, an intel/amd laptop with 128GB RAM and 16GB VRAM can run the IQ4_XS quant and decode about 4-7 token/s, depending on RAM speed and context size.
Using llama.cpp, the decoding speed is about half of that.
Mac with 128GB RAM should be able to run the Q3 quant, with faster decoding speed but slower prefilling speed.
Yeah I haven't seen anyone actually believe this either, but it did take a good few years between the flat Earth joke being popular on Reddit, and the resulting community of true believers arising.
On the other hand it's even less plausible than flat Earth so you'd have to be particularly stupid to believe it; not just very very stupid.
reply