I would argue that’s a driver to my point. How many people are never going to visit the source website when Llama can give me a detailed summary of what I need in a few hundred milliseconds? I would consider that in the same category of forgotten. I could’ve been more clear in my other comment.
If a website is not financially dependent on search traffic, they can block all scrapers with a paywall, and their content will be missing from generic LLMs.
If a website is financially dependent on search traffic, they can go out of business due to loss of traffic to LLMs, and their content will disappear everywhere.
If the majority of websites fall into the latter category, LLMs would be left with old/archive longform content, plus micro content from social media.
If social media (e.g. X.AI) takes their data private for vertical integration with payments and internal LLM, their content will be missing from generic LLMs.
