As a user who's constantly clicking on the crosswalk or storefront images you can't help but to think that you're essentially working for free training Google's machine learning models by providing them with supervised data points.
That's what ReCaptcha always was... it was originally a known and another unsure text blurb from scanned books/text documents. Now it's street signs etc.
It's actually pretty interesting to see how the captchas have evolved as, presumably, Google decides "OK, we have enough data to consistently identify this thing" and moves on to the next challenge.
I recall the modern (non-text) captchas used to be cars pretty much every time. Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions. Then crosswalks and store fronts became quite common, eventually with the same kinds of noise distorting images. Now I've started seeing things like buses, bridges, motorcycles, bicycles, etc. It feels like they've finished getting enough data for improving Google Maps and have begun moving towards collecting data for their self-driving car projects.
Machines have been able to beat captchas for years and years. The point is that it costs money and that's good enough to prevent free and scalable abuse.
Those really grainy images, they always seemed like they had to have noise added on purpose? It was like really badly processed film grain, but if the images were enlargements then wouldn't they be pixelated?
Stuff you get now often requires cultural information, like "sidewalk" isn't a cross-cultural name, I'd guess almost everyone knows it, but meh. What classes as a store, is a lawyers office a store? Also, I seem to recall I had "click on all minivans"?? Not sure what one of those is, nor really what is classed as a car in USA, is an MPV a car [I'd guess that's what a minivan is?]? Do pedestrian crossing lights (green/red man) count as [part of] traffic lights? I've often wanted a short description of the locus of the terms they're using. Of course it never tells you if you failed, just gives you a further captcha, which it might have done anyway.
The fire hydrant one always annoyed me - in many countries stand-up fire hydrants are nonexistent or at least look very different. I guess they're banking on people having seen enough American movies/TV shows?
In the UK we call them fire hydrants and they’re typically placed underground with a small metal cover in the middle of the road and a sign on the pavement (sidewalk for Americans) saying where it is.
Yeah, a lot of the storefront ones are just plan hit things randomly until it lets you through - how am I supposed to know if a building with some writing on it in Korean is a store or something else? Are you supposed to include the poles in traffic lights or not?
The V2 was just annoyingly badly designed because the questions were badly put.
I think it might be the same when they switch to other types of objects (like crosswalks or bikes). Someone's model got too good, so they had to change to something else. I also get the impression that they add delays to the tile refresh before they do that.
I suspect Google now uses robots to generate captchas for humans, under the assumption their image recognizers are far better than anyone else's. They already have some very well ones for other products (self driving cars, street view) and lots of street-level city imagery. That would explain why their captchas are so difficult for humans to solve -- they're testing if you see things like their "AI," not like other humans.
>I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha
I was thinking it was trying to dirty up the image just like the lenses on cameras get dirty. What happens to the image recognition when there's water spots, dirt, mud, etc on the lens that keeps parts of the image obscured?
what would be the training data value in getting humans to classify images you have clean versions of, after applying simulated noise?
If you have the clean version of the image, you need to get that classified by a human - then you can throw noisy versions of it into the training set for your AI. You don’t need to ask a human, hey, I added noise to a picture of a yield sign. Is it still a picture of a yield sign?
With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot.
You can only conclude that recaptcha is a ml training exercise.
> With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot.
The article explains that this is part of what reCAPTCHA does, e.g.:
> Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. When you click that checkbox, Google reviews your browser history to see if it looks convincingly human.
But your point is otherwise right in that it's used for ML training, which Google admits as another commenter pointed out.
> Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things.
Human [n]: Entity that uses Google®-brand services.
Maybe, maybe not. From time to time google forces me to prove I'm not a robot because of unusual search queries. Either my queries are unusual or I have an infected computer. The thing is, it also happens to me on my iPhone.
Or, maybe they feel I'm not pulling my own weight, seeing as I rarely ever click on adds. They probably need more monkeys to feed the beast so I get selected to train their AI beast.
Edit: It could also be that I'm always running on incognito mode.
And I find it equally interesting to think about what happened to Google Books project after half the world was entering text into captchas perhaps close to a decade? Is the project still going on?
Helping scan all the world's books as part of a plan to make them freely available to all is much easier to get behind than tuning map objects for their self driving cars. Particularly when most books that were scanned never, ever showed up on Google books.
> Particularly when most books that were scanned never, ever showed up on Google books.
If one looks at the history of Google Books one can see that they started with big ambitions, but hit copyright in quite intensive ways. That also changed their approach to other projects. Clearing all rights internationally isn't easy.
And if they succeed to much they can monopolize transportation and thus mobility. Not a power I want to have in a private company. Especially not in a company from outside my country's jurisdiction. Where I can't have an impact via democratic law making process.
> Especially not in a company from outside my country's jurisdiction. Where I can't have an impact via democratic law making process.
This is false. EU governments have already placed significant restrictions and fines upon US tech companies in the past. There is no reason to believe that they won't be able to again.
Those fines are the perfect representation of the lack of control EU has over US tech giants. They are essentially opaque and impossible to inflence through normal regulatory and political channels so the only options are the big guns. You can be sure Google is not going to heavily invest in the EU tech sector under such adversarial set-up. There's a subtle blackmail here: we push the envelope as far as it goes and the EU can choose to submit or risk technological backwardness.
It's a great situation for the US economy but a very bad strategical position for Europe.
Not if Google has collected enough other identifying information about you to determine whether or not you're a robot.
But I'm kinda hoping that the reason I keep having to identify cars and store fronts is that my refusal of third-party cookies is causing them to have no idea who I am. But that might be the optimistic view.
In any case, I wouldn't mind if sites stopped using recaptcha.
Identifying text for public domain works that Google is making available online is much different than forcing the web to train their models for their profit.
they stopped using the text because some forum campaign that promoted typing cursewords instead of the unkown word. they probably started showing cursewords on the rendered search highligths on google books.
there was a decent write up from a whitehat showing the damage, but I can't find it
I can't imagine some forum has enough traffic to meaningfully screw up their data, and they don't tell you which of the two words is the unknown word, so you're just going to fail a lot doing that.
This was extremely common on 4chan maybe 7 or so years back I can't remember the exact year and it worked for a very long time before anything was done about it and everyone knew to do it. Google asked for two words and presented them as two different fonts. The real word that needed transcribing was always identifiable and you could just write what you wanted as long as you got the second test word correct. Much fun was had.
So what's going to happen if you write "nigger" that you're so unwilling to write it? It's not like you'd be calling someone a nigger, you'd be using it in a clearly informative manner,
> they stopped using the text because some forum campaign that promoted typing cursewords instead of the unkown word. they probably started showing cursewords on the rendered search highligths on google books.
If I remember correctly, Google later on also sometimes showed two "known" words or, if they had actual other evidence that you are human, two unknown words.
> I'm doubtful about any good solution for having computers decide whether we are sufficiently human.
Two questions:
- Couldn't a computer just temporarily hire a human to prove there is a human involved?
- Why are we using recaptcha or verifying humanity anyway? I understand stopping spam, scams, and fraud, but scraping already public data doesn't present significant harm.
Yeah, it shouldn't be used to limit public data. The main use is to prevent spammers from spamming fora or registering thousands of throwaway email addresses.
I've been thinking about this a lot lately. Where is our compensation? It's our time and brain power training Google's AI that will one day be sold back to us. I'm really not into this.
Because Google can extract value from captchas, it makes world-class captchas and bot detection AI available to every webmaster for free. I don't know what that level of service would otherwise cost, but it almost certainly wouldn't be affordable for low-traffic blogs and the like, which would end up vulnerable using weaker captchas or trying to roll their own. Everywhere else the cost would just get passed on to users.
I don't love the compromise of paying for things with my data or by training Google's AI, but it's hard to say users aren't getting anything out of it. That said, I do miss the old reCaptcha.
> it almost certainly wouldn't be affordable for low-traffic blogs and the like
Very few low-traffic blogs that I see use (or need) CAPTCHAs. I know that the ones I run don't.
> I don't love the compromise of paying for things with my data or by training Google's AI, but it's hard to say users aren't getting anything out of it.
I don't think they are getting much, if anything out of it -- aside from being increasingly punished for defending themselves against being spied on by Google.
Often a trivial non-standard thing like "what's the name of the author" works well enough. Especially outside the English language. Spammers won't spend the time to bother adopting their scripts for that.
If this somple thing comes from a popular WordPress plugin the equation for the spammer changes, of course.
There's certainly a period of time where that solution is sufficient as it stops the lowest level of drive-by <form> spam.
But it also sucks the first day you get an attacker who solves it once and then spams you thousands of times.
Modern spam tools are pretty impressive these days and minimize the targeted work the human spammer needs to do in these cases. In the early 2000s, you could set a custom question and then assume no attacker is going to manually code for your little blog.
But even in 2008 I was using spam software (out of curiosity) where you could import a massive blog list, and it would pause spamjobs with failed comment submissions, let you pencil in a value for this unknown field, and then click resume.
You could also choose other actions for that field like "prompt me each time" and sit at your computer multiplexing your labor across hundreds of blogs. And that was pretty polished ten years ago.
I do this. Funnily enough one of the reasons I did it was because Google’s spam filters gave me too many false positives and my gmail account attracted enough spam that sorting through manually was a pain.
Has it been a good experience for you? What are you using?
My point was just that even if something is provided to the customer for free, doesn't mean it's easy to produce. That causes a lot of the issues my non-tech friends have with understanding the scope of work. Just because social media is free and easy to set up as a customer doesn't mean developing a social media is easy at all.
I’ve been using exim and dovecot with rspamd for spam filtering. Have two VPSs on different providers to provide MX backup properly (they’re cheap these days and for low traffic I don’t need much more than the smallest VPS). I do DKIM and SPF but not DMARC and it gets through gmails spam filter fine and passes the various other tests you can find online. Took a while to set up right (in the end I found the best route to predictability to be writing my own exim config file rather than using someone else’s template) but pretty simple to run after that - there’s some effort to make sure I keep up to date with security patches and monitor log files for anything untoward but it’s relatively small. Using letsencrypt certs so email clients have been relatively simple to set up.
Overall it’s been a good experience. I run into a few sites which when I send to them classify my email as spam or grey list my sending IP so mail doesn’t get through quickly but then I used to have the same spam problem with some sites running my own domain through google apps.
And there's going to be a lot of discussion of the idea at the RadicalxChange conference in March (https://radicalxchange.org/), including with Jaron himself as well as the book authors. (Disclosure: I do the conference website as a volunteer).
Are you kidding? Your compensation is all the free apps you get (Gmail, Maps, etc.) You’ll rarely see a Captcha for a paid product once they have your cc info.
It's actually not that hard, assuming you control your form generation. Bots usually fill in fields using the actual field name - not the label the user sees. So provide a field labelled "Age" but named "email", and simply check it contains digits. If it's got an email address in it, it's a bot.
Labels can also be obfuscated with javascript, replacing the raw HTML "Email" with "Age"on page load. Getting this right will require the bot to parse both HTML and JS, and we can force them to handle CSS too. Add a "zip" field, and hide it with complex CSS rules. If it contains a zip code, it's a bot.
If you're really paranoid, randomise combinations of distinguishable fields (name, email, phone, age and hidden fields) every time you generate the form, so even if a bot herder manually maps names to fields one time, it'll fail the next. At this stage it'll be cheaper for the bot herder to use Mechanical Turk, after which even Google's captcha is compromised.
>So provide a field labelled "Age" but named "email", and simply check it contains digits. If it's got an email address in it, it's a bot.
Or a blind user who might actually rely on both labels and names. That's a bit like what arxiv does, they have hidden links that ban your ip when you crawl, but the links aren't hidden for AT users. I got myself banned that way once.
I've found that the tech industry often is. Trying to get managers to set aside time to iron out accessibility issues is like pulling teeth. Trying to get other developers to take it seriously is almost as bad. Often you count yourself lucky if the bare legal minimum is implemented.
Accessibility is very important, and if accessibility features are implemented well they'll often be useful even to people without disabilities, but do any CS/SE or code bootcamp programs take the topic seriously? I'm sure it must be taught somewhere, but it doesn't seem to be common at all. Can you even imagine 21st century university architecture department that didn't cover ADA compliance? That'd be unthinkable.
wonder if someone is collecting actual data instead of listening to udfalkso, the google sales rep here.
maybe to save them a few $ from bots and spam (bandwidth and storage is very cheap today) they might be losing new users by the thousands (and traffic acquisition is far more expensive than the formers)
Off the shelf spam software like Xrumer[0] has been cracking those captchas for 10+ years.
Recaptcha isn't obnoxious for fun, it's obnoxious because this is the state of the arms race right now. There's also the challenge of creating a captcha that allows blind people in.
Yeah, that is what annoys me. “Thanks for paying us to use our product. Now do free work for us for the privilege of using the product you already paid for!”
> Your compensation is access to the website that uses Recaptcha
Which is one of the reasons why the presence of reCAPTCHA is strong push to avoid that site.
> since you're here and HN uses Recaptcha on its register/login form, it seems like the compensation was adequate.
Perhaps so. I don't remember doing a CAPTCHA to sign up, but I don't dispute that I did it. However, I've never been presented with one after signup. If I was, I wouldn't be here.
I don't really understand the case where you'd use this.
First, it seems tacky scrounging for peanuts from the users' captcha work. Or it's like a product/services website showing Adsense ads. It's a cheapening message to send.
Second, since you make more money from more captcha volume, you're incentivized to maximize your use of captcha which is at odds with every complaint in this comments section about captcha. Most sites only use captcha to gate low-volume actions like register/login (e.g. HN).
They created their own Ethereum token too which always puts a bad taste in my mouth these days.
Finally, it doesn't address the upstream complaint that someone else is profiting off the user's "work" rather than the user. Though I don't find that complaint very reasonable. And a tiny fraction of a cent sounds about right. The truth is that users benefit from anti-abuse systems. The number of bots that HN's recaptcha on register/login has stopped is worth that tiny fraction of a cent to most users.
Is it somehow less tacky to give that value away to Google for free?
Sites can set the difficulty level necessary for their application. Some are under continual targeted attack, others are mainly keeping out rogue automated spambots from their comments section.
The user is typically getting a free service, a better site experience due to less bot traffic, or both. I think sharing the value of their work with the website is a fair deal.
As for using blockchain tech for ledger functions, that is all under the hood: websites can cash out to dollars as they prefer.
(disclosure: work on bot detection at hCaptcha.com)
> Is it somehow less tacky to give that value away to Google for free?
Yes, mainly because we're talking about fractions of cents. Also, it's not for free; the website and its users get a good anti-abuse measure in return.
There's a big difference between something that cannot make money and something that makes pennies for the site. But, to be fair, 99.9% of users aren't going to notice the difference in captcha branding either way unlike my example of a banner ad on a retail site.
My main reaction is that the UX incentive to minimize user exposure to captchas seems to work against the primary pull of using hcaptcha in the first place.
Though one site I can think of that has a captcha behind every action (every post) is 4chan. Maybe you can get them on hcaptcha one day. It would at least help you test your tagging system against vandalism. :)
I assume you're talking about ads. But you're bidding at least cents on Adwords and making at least cents on Adsense. In Adsense's hey day, I had relatively low-volume sites making over $1 per click and paying my rent in lucrative niches.
I didn't find any pricing examples on hcaptcha's website. For all I know, people are bidding 5 cents per image.
Anyways, I definitely want to see more serious contenders in the captcha space so that we all aren't contributing to Google's middle-manning of the entire internet, and I'd like to try hcaptcha even out of curiosity.
Your compensation is that you get to use the website, and the website's compensation for putting up a gate for their users is that they get to keep the bots out.
When you search for an address on google maps, that little tiny house number on the house was once a captcha image and now google knows that number so it can take you to the exact location on a map when you search for that number.
Everyone helps train the machine so when they want something from the machine then the machine is better at finding what they asked for. That seems pretty democratic to me.
So you want compensation because your data is used along with millions of others to train an algorithm to distinguish if a bot or a real human to provide a service to you? Nice.
The Cascade Bicycle Club in the Pacific Northwest threw me into one of these multi-minute Captchell vortexes when I tried to log in to my years-old account to renew my membership and register for one of their organized rides. I was already on the fence as to whether it would be worth paying to do these rides that I have already done several times over the years. That (ironically) dehumanizing experience pushed me over the edge. I didn't complete the Captchas, didn't log in, didn't renew my membership, and didn't register for anything this year.
More and more frequently when presented with a captcha, I've been deciding I don't care enough about whatever it was to want to exchange robot training for access. Especially if they pull that shit after I've spent effort on something (a comment, say) - I will absolutely walk away and not come back.
One solution is to remove Google out of your life as best as possible.
Personally I now use...
- iCloud.com instead of Gmail
- DDG for search though I do have to !g like 20 to 30% of the time for things like driving directions (from X point to Y point), local movie times nearby and flights.
I still use
- YouTube
- Google Maps some as its great for getting distance between X and Y
- Google News (is there a better substitute)
- Google Photos (is there anything that compares)
Hoping in time to rely a ton less on Google products.
I'm in the same boat, friend. Expelling Google from my life.
Apple Maps works for me. I appreciate that's not the case for everyone, but it's come a LONG way.
I sincerely use Apple news (on iOS) and have been loving it, but appreciate it's not for everyone's use case.
Google photos.. yeah wow. There really isn't much like it. I've resigned to storing my photos myself on a private server and slowly making albums/things come together. But I have to NOT use google photos. It's too scary.
Gmail was easy
Youtube I use a fake gmail account that's not linked to me in the slightest and only use it on 1 iPad, else not logged in.
It's a quest. But I'll get there. Someone really ought to make a Google Photos competitor though, there's nothing that has the same level of polish right now.
Not a solution in this case - the problem is that companies like Cloudflare use these captchas to slow down your visit if you are using Tor or a VPN. In fact being logged in to google probably helps you bypass these checks
I suspect you could replace the entire mess described in the post with: what's the logged in account's spaminess? followed by, what are the doubleclick cookies' spaminess?
You could further approximate that with: "How much does Google's AI think this human's time is worth in future revenue?"
I for one intentionally inject errors into their image classifier until it lets me in anyway.
I always try and click one or two extra boxes that are wrong. Sure I sometimes have to go through and confirm extra images, but you know what? I don't work for free and so I'm doing my small part to bugger up Google's data set.
I've recently made a discovery that pleases my petty side.
You know how they usually give you several questions to solve, even if you're quite convinced you solved a question correctly?
Turns out if you click randomly, they keep showing you new questions as well. If, after a handful of purposely wrong answers, you answer one correctly, they let you through.
I now purposely mess up the answers a few times. It seems neither slower nor faster than actually taking the time to do it right, but it takes less mental load, and it makes me not feel like doing slave labour for a machine.
I'm now wondering if the fact that I block third-party cookies is the reason I always have to identify cars and store fronts. Maybe I should just avoid sites that use recaptcha.
For a while there they kept saying I was wrong and it would effectively lock me out of some accounts that required it. Recently it’s stopped but it’s super annoying how ambiguous it is.
