Paper Reading #6

Comments:
Comment #1
Comment #2
Reference:
Title: Who are the Crowdworkers? Shifting Demographics in Mechanical Turk
Authors: Joel Ross, Andrew Zaldivar, Lilly Irani, Bill Tomlinson, M. Six Silberman
Venue: CHI 2010, Atlanta, Georgia
Summary:
This paper describes Amazon Mechanical Turk, a system that distributes tasks to anonymous workers for completion. The workers are usually compensated $0.01-0.10 per task. Sometimes a few dollars will be given for longer tasks. The workers are referred to as "Turkers." The system's name is a reference to Wolfgang von Kempelen's automaton.

The demographics of the workers has been changing the last two years. Previously 60% of the Turkers were female, while now women compose 52% of the Turkers. Additionally, in 2008 76% of the Turkers were from the US, and in November 2009 that number was down to 56%. The average age of the Turkers is lower as well.

The group posted the survey as an HIT on the site, and payed $0.10 for completion. However, they didn't require the ones who completed the survey to have a 95% accepted work rate, as Amazon does. This poses the concern that people may not have been honest and just said what they believed the surveyors were looking for to ensure payment.
Discussion:
This article was pretty lame. There was no real interface design discussed. It did explain how some people use an existing system, but didn't offer any improvements or real criticisms. This article was basically an statistical analysis of the system without discussing the efficiency of it or providing feedback. The entire paper was explaining the results of the survey they administered. Heck, I could post a survey and create some graphs, and then discuss the results. I just did not like this article very much. I hadn't heard of the Mechanical Turk system, which sounds pretty cool, but that was the extent of my interest.