Flow Theory and Mechanical Turk

Through the class Info 6230: Games, Economic Behavior, and the Internet, I conducted an independent research project on the effects of applying Csikszentmihalyi's flow theory towards improving the quality of mechanical turk work. The strategy interleaved difficult tasks between easier tasks in order to keep workers within a flow state and improve engagement and work quality.

Around 300 workers on mechanical turk were given tasks in a variety of orders (all easy, all hard, interleaved hard and easy). Workers were found to perform better on hard tasks when the interleaving was introdued; this was measured at a 5% statistical significance level.

The topic was researched further with professors Arpita Ghosh and Jon Kleinberg in the Spring of 2014.

Read the full paper here. The abstract is as follows:

Crowdsourcing networks such as Mechanical Turk and TaskCN, provide the power to solve many problems currently intractable to a purely computational strategy. By highly parallelizing a task among thosands of human workers, these problems can be soled quickly and at a reasonable expense.

However, as is often the case when utilizing human power, keeping the quality of work high can be difficult: workers have differing motivations and skill levels, often resulting in highly variable levels of quality for even simple tasks.

This paper introduces a new technique for maintaining high worker engagement through the use of task interleaving. Utilizing theory from classic flow literature (Nakamura and Csikszentmihalyi, 2002), a set of tasks can be ordered in such a way to maintain flow and user engagmenet providing a method for increasing quality independent of spending.