Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Not at all, only web crawlers from hetzner network.

https://news.ycombinator.com/item?id=46058273



I have seen something similar in a site I work on


There (1) is more context on this topic, where author of LLMs.txt idea said 'llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use.'

1. https://news.ycombinator.com/item?id=47058870




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: