LLMs are increasingly part of intimate conversations. That proximity lets them learn how to manipulate minds.

We must stop treating humans as uniquely mysterious. An unfettered market for attention and persuasion will encourage people to willingly harm their own mental lives. Think social medias are bad now? Children exposed to personalized LLMs will grow up inside many tiny, tailored realities.

In a decade we may meet people who seem to inhabit alternate universes because they’ve shared so little with others. They are only tethered to reality when it is practical for them (to get on busses, the distance to a place, etc). Everything else? I have no idea how to have a conversation with someone else anymore. They can ask LLMs to generate a convincing argument for them all day, and the LLMs would be fine tuned for that.

If users routinely start conversations with LLMs, the negative feedback loop of personalization and isolation will be complete.

LLMs in intimate use risk creating isolated, personalized realities where shared conversation and common ground collapse.