The next step: turn it into a game and get it done for free. Even better, put a skinnerian ux around it and charge for the pleasure of solving picture puzzles ;)
Another possible way to do this is present the Turker with two vertical pieces already placed together. Ask them a yes/no question: do these two images match?
Iterate intelligently over your favorite sorting algorithm until you've placed all the images!
Note: requires more Turkers but each answer could be worth much less, maybe around $0.01
The lower the cost of the HIT (and the easier the question) the more you want to make sure you're screening for bots.
Make sure you have a gold standard question. For every HIT ask two questions; ask the legitimate question that you need answered and then on top of that, ask a simple "gold-standard" question that will determine whether the answer submitted is a bot or not.
For the gold standard, I like to choose two numbers, and store the sum of their values. Along with your HIT question, be sure to ask the Turker the value of their sum.
E.g. For a survey you ask the Turker "What is your favorite color?" But then along with that, you ask them "What is four plus fve?"
Be sure to misspell one value (e.g. remove a letter, etc). If the Turker doesn't respond correctly to the "gold standard" question, programmatically reject their response.
But then I think you are giving up the main advantage of using mechanical turk over just roughly matching pixels. It is possible matching pixels will get the wrong answer where a human, with the context of being human, will know what the image is supposed to be and get the write answer, captcha style.
Example: an iron fence with a scenic view behind and where the slices are too small for someone to match only two.
What about doing the same thing, but fuzzing a few pixels from each edge to avoid pixel distance matching? You could even make the edges just plain black for a few pixels from each edge. The human eye would be able to complete the image (it would just look like it has black bars), but I'm guessing it would be enough to prevent a simple algorithmic solution.
Edit: This is essentially the same idea as the iron fence posted above.
You can't use merge sort or any other standard sorting algorithm in this case. All these algorithms assume you can compare any two elements. To use a sorting algorithm, you would need to be able to say for any two images which is further left.
This is a really neat hack but How does the script know whether or not the solution was correct? Have they shared a real solution that solves this problem completley algorithmically, without human involvement? I could not find it.
This is probably the first time I've ever cares about "crowdsourcing" or his ugly brother-in-law, "cloudsourcing". However, I can't help but imagine why they want a solution for this. Seems like Cold Wat-era spook stuff.
It was a hiring contest/game to get people thinking about Instagram and find some talent, rather then a search for better system like Netflix's contest.
Probably not, as then you have to play around with blending two images and (un?)distorting them to match each other's shape. That's the hard part - people tend to take pictures in order rather than randomly.
If you're trying to imply that I "stole" the comment, I posted it in the article comments after posting it here (note the usernames). Figured it might spark some discussion for people not coming from HN.
I know that, I saw that it was you. I also don't have a problem with it, it just caused me to do a double take and go check the site to see if my deja vu was real. I thought I would mention that because I doubt I was the only one who had that experience.
OT: For some reason the Recollect bar that slides in from the top when I scroll down the page is very distracting. It seems to violate the expectation that, when scrolling downward, page content should only be moving upward. It also feels like it's robbing me of readable screen area. I suspect I'd be much less bothered by a position: fixed header that was always present.
This is a very cool way to use the Amazon Mechanical Turk!
I am 99% sure that the image (at least the one that I saw) on the page is of Tokyo's Shibuya station. And the tall white and black building on the right 1/3 of the picture? That's the Cross Tower building, housing Amazon's Tokyo office.
This photo had to have been taken from one of the upper floors of the Cerulean Tower hotel.
Very clever sir! I've used the Mechanical Turk for a couple of interesting tasks, lowering prices to $0.01 and having Turkers complete those tasks in less than 2 minutes which is pretty awesome! It's very interesting how we've started to create some kind of relationship with Turkers, since we've sent over 1M HITs since we started using it.
