> All of this comes from there being no universal way to prove you are a human on the Internet. If somebody were to invent a physical device (think YubiKey) that atttested that your activity is human without it being usable to identify/track you, we might have a shot at solving this without CAPTCHAs.
Which is good. That's a desirable property. The distinction isn't available without also allowing fingerprinting.
Further, the line between bot and user-agent is not perfectly clear. Something like cost-based attestation where humans and bots are treated equally is ideal.
> And before somebody makes the argument of "but that's centralised, big brother, blah blah whatever bullshit", let me remind you that every payment you make goes through either Mastercard or Visa.
Is it? That's Cloudflare's whole selling point - keep the bots out. I can understand from a hacker perspective wanting bots to be able to roam the Internet as freely as people but that causes massive headaches for sysadmins, SREs, and DevOps. robots.txt is no good because it's opt-in.
Cloudflare's decidedly _not_ about keeping the bots out. It's about keeping out malicious traffic. This seems like a tautology, but I'll explain why they are not the same: When I hit refresh in my RSS client and it GETs 250 different servers, on my behalf, is that a user agent or bot activity? How are you going to differentiate the two by their behavior? Some bots are let in, on purpose, like search engine crawlers. Some users are kept out, on purpose, because they use anonymity tools.
Since we don't have chips that detect one's heart's intentions yet, the best we can do is treat bots and user agents the same, and address the problem of malicious activity in other ways. This can be rate limiting, paying per request (i.e. hashcash) or other mechanisms I don't have top of mind. But bot=deny and user=allow is not what Cloudflare does or seeks to do.
Which is good. That's a desirable property. The distinction isn't available without also allowing fingerprinting. Further, the line between bot and user-agent is not perfectly clear. Something like cost-based attestation where humans and bots are treated equally is ideal.
> And before somebody makes the argument of "but that's centralised, big brother, blah blah whatever bullshit", let me remind you that every payment you make goes through either Mastercard or Visa.
That's an even bigger problem!