For each token generated, you only send one token’s worth between layers; the pr...

		remexre 48 days ago \| parent \| context \| favorite \| on: Big GPUs don't need big PCs For each token generated, you only send one token’s worth between layers; the previous tokens are in the KV cache.