>At what point do we cross a threshold here and encounter a non-trivial number of people who argue that a sufficiently large model actually is a brain, and therefore deserves rights?

There are two ways of resolving this conundrum. You either give rights to computer programs, or take away the rights of people. Which option sounds more likely to you?