This seems really interesting but I can't quite figure out if this is like a SaaS product or an OSS library? The code sample seems to indicate that it uses some sort of "client" to send the document somewhere and then wait to retrieve it later.

But the home page doesn't indicate any sort of sign up or pricing.

So I'm a little confused.

edit Ok I found a sign up flow, but the verification email never came :(

> It's not clear where (and maybe not easily accessible) to get these numbers at -exactly- the moment they release.

This is all publicly available to everyone at the exact same time. How do you think the market is moving in response, if this info is not available?

Because the website doesn't immediately update. The blockchain approaches something dynamic, where the update is broadcast to everyone at the same time, not just broadcast to those who know where to ask for it at the right time.

It seems similar to people who harvest the URL of a product before launch, so they don't have to wait for the slow website.

I guess. It doesn't seem like people who want/need this data in instant real-time are having trouble getting it.

> AI thrives in messy, open-ended spaces

Isn't the exact opposite of what AI is good at? That sounds like a great way to get tons of hallucinations. Every "how to AI good" guide I read is all about providing it with ample structure and narrow instructions.

To be reductive about the example of a malleable task tracker (e.g. Notion), if the LLM correctly interprets the user's prompt, there's about 20-30 useful buttons in the interface, and there's a sequence of those buttons that achieves the user's desired outcome. It's open-ended in terms of what you make, but the set of available actions is fairly tight, and the problem is finding a sequence of those actions.

They are obnoxious and they used to be 10x more obnoxious.

We have all of those things in the states too. Just not ubiquitous.

We don't have much of it, not compared to Europe or Australia. This is a solved problem, but we don't want to solve it.

My personal experience is that OpenAI's crawler was hitting a very, very low traffic website I manage 10s of 1000s of times a minute non-stop. I had to block it from Cloudflare.

Where is caching breaking so badly that this is happening? Are OpenAI failing to use etags or honour cache validity?

Their crawler is vibe-coded.

Same here.

I run a very small browser game (~120 weekly users currently), and until I put its Wiki (utterly uninteresting to anyone who doesn't already play the game) behind a login-wall, the bots were causing massive amounts of spurious traffic. Due to some of the Wiki's data coming live from the game through external data feeds, the deluge of bots actually managed to crash the game several times, necessitating a restart of the MariaDB process.

Wikis seems to attract AI bots like crazy, especially the bad kind that will attempt any type of cache invalidation available to them.

OpenAI straight up DoSed a site I manage for my in-laws a few months ago.

What is it about? I'm curious what kinds of things people ask that floods sites.

The site is about a particular type of pipeline cleaning (think water/oil pipelines). I am certain that nobody was asking about this particular site or even the industry its in 15,000 times a minute 24 hours a day.

It's much more likely that their crawler is just garbage and got stuck into some kind of loop requesting my domain.

It's common to see them get stuck in a loop on online stores trying every combination of product filter over and over.

Even Googlebot has to be told to not crawl particular querystrings, but the AI crawlers are worse.

Not an online store just a bog standard Wordpress site

I suppose that they just keep referring to the website in their chats, and probably they have selected the search function, so before every reply, the crawler hits the website

Ah nice. This whole debate died with Covid but I guess there’s enough RTO happening to re ignite it.

I worked in one in 2008-09. Reception area and then a bunch of hallways with 20-30 individual offices and conference rooms. It was nice.

That's not really how that works in the corporate/big tech world. It's not as though Meta set out and said "Ok we're going to hire exactly 150 AI engineers and that will be our team and then we'll immediately freeze our recruiting efforts".