7B & 13B can be ran on a M1 Air with 16G memory: 7B uses about 4.5G max & runs a... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

boesboes on March 11, 2023 | parent | context | favorite | on: Running LLaMA 7B on a 64GB M2 MacBook Pro with Lla...

7B & 13B can be ran on a M1 Air with 16G memory:

7B uses about 4.5G max & runs at 203.38 ms per token, 13B about 8G and does 396.58 ms per token.

30B needs about 20G and basically hangs due to swapping i guess with 16G.

arvinsim on March 21, 2023 [–]

Just the comment that I was looking for. Guess I can run it on my M1 Pro 32GB model

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact