That is more than an order of magnitude better than my experience; I get around ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pocketarc on Sept 12, 2023 \| parent \| context \| favorite \| on: Fine-tune your own Llama 2 to replace GPT-3.5/4 That is more than an order of magnitude better than my experience; I get around 2 t/s with similar hardware. I had also seen others reporting similar figures to mine so I assumed it was normal. Is there a secret to what you're doing?

ramesh31 on Sept 12, 2023 [–]

>Is there a secret to what you're doing?

Core speed and memory bandwidth matter a lot. This is on a Ryzen 7950 with DDR5.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact