Amazon combines mobile with Mechanical Turk

(Warning: if you already know about Amazon’s Mechanical Turk, this is probably a post for you to skip. Just take away  the news that Amazon’s new iPhone app has a great camera feature that makes use of mechanical turk)

Here’s something I wanted to point out today (I found the news of the app itself here on NMA) – not because it’s another iPhone app, but because it illustrates one use of some  fundamental differences in economic models possible because of the internet and unlimited scaleability through bits & bytes.

The background:

For those who don’t know Amazon don’t just do books and e-tailing :

• Quite some time ago they realised that they could take their prowess in computing, and their sheer size and economies of scale, open them up to small to medium (to large) companies who had a need for on-demand storage and processing power, and start making additional revenue that way.
• That’s where Amazon S3 and EC2 have come from – storage and elastic computing products respectively. S3 particularly is very widely used amongst new and not so new web start-ups as it provides them a simple, scaleable, cheap way to, for example, hold a bunch of user images or data (Twitter for example does some of its storage with Amazon).

In a similar way, Amazon have an offering called Mechanical Turk:

• Like S3 and EC2 above, it came from their own needs but is now available to any developer that wants to make use of it.
• What is Mechanical Turk?  It’s a system that lets you send out little tasks to humans spread all/anywhere over the world to complete. These tasks are things that computers just cannot handle yet - things like image recognition, or transcription of audio files, or de-duping of complicated data such as book titles with different ISBNs.
• You get to set a price for each task and the workers get to pick and choose which tasks to work on.
• The results of the tasks can then be fed back into your own programme or database or site.

(You can find out more about Mechanical Turk at Amazon’s website)

Amazon’s own iPhone app below is one awesome way this is being put to use.

Using Mechanical Turk in an iPhone app

Amazon have just released an iPhone app here in the UK. It has the usual search and buying functionality that you’d expect; account history etc etc.

But it has a cool extra functionality:  You can take a photo of something you want to “remember”, and it will find that product in its catalogue, so you can purchase right then and there.

I just tried 3 different products, and got the answer back for all 3 in well under 4 mins.   (They’re calling the feature “Amazon Remembers”.)

How does it work?

• Automatically Amazon’s system approximates the image to a shortlist of images from potential matching products (maybe 3 or 4 I would guess, depending on how complex the image is)
• Using Mechanical Turk, my exact image and the shortlist of potential match candidates is added to the jobs queue. This jobs queue is the thing that 1000s of people keep an eye on – they select any job they like to work on. Their task for this particular job is to pick which of the shortlist is in fact the product I took a photo of.
• The answer is then sent back to Amazon, and to my phone.
• Because of the scale of all of this, there’s every probablity in reality an extra “safeguard” that Amazon implements, to increase accuracy of the result sent to me:  my picture will in fact be duplicated as a number of different jobs (who knows the number, but the economics of it all mean it could easily be 10 or 20  or 30 times). The Amazon automated system will then see how much all the different answers from the different jobs are in agreement, and send me the one that fits their criteria of “right” (so it might be the most popular one, or the one that 90% of their task responses said).