> It’s just really clear that a giant text averaging machine can only go so far
It's not really a text averaging machine, it's a pattern matching machine.
Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.
it is a statistical model. If everyone is saying X and is wrong, and one guy says Y and is right, the llm will bit out X. Because that is the most probable thing in the dataset. It literally is a text averaging machine
It's not really a text averaging machine, it's a pattern matching machine.
Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.