The negative quality impact of quantization is more pronounced for smaller model... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Palmik 12 months ago \| parent \| context \| favorite \| on: DeepScaleR: Surpassing O1-Preview with a 1.5B Mode... The negative quality impact of quantization is more pronounced for smaller models [0], so I'm surprised this tiny quant works at all. [0] or rather models closer to saturation, which is a function of model params and amount of training

simonw 12 months ago [–]

Yeah I'm very impressed that the tiniest quant version appears partially usable... especially now I've replicated that same S, T, R, A, W, B, E, R, F, U, R, Y bug on a much larger one: https://news.ycombinator.com/item?id=43018494

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact