Colloquium: Artificial Intelligence for Artificial Artificial Intelligence

Peng Dai, Graduate Student, University of Washington Computer Science

Venue:  220K CRMS

Time: 4:00-5:00pm

Host: Professor Goldsmith

Abstract:                                                                          

Crowd-sourcing is a recent framework in which human intelligence tasks are outsourced to a crowd of unknown people ("workers") as an open call (e.g., on Amazon's Mechanical Turk). It is also known as "artificial artificial intelligence". Crowd-sourcing has become immensely popular with hoards of employers ("requesters"), who use it to solve a wide variety of jobs, such as dictation transcription, content screening, translation, information extraction, etc. In order to achieve quality results, requesters often subdivide a large task into a chain of bite-sized subtasks that are combined into a complex, iterative workflow in which workers check and improve each other's results.                                                                            
                                                                                    
In this talk, I will introduce a planner, TURKONTROL, which formulates workflow control as a decision-theoretic optimization problem, trading off the implicit quality of a solution artifact against the cost for workers to achieve it. We learn the models from real data, and demonstrate that the dynamic workflow, generated by the decision-theoretic agent based on the model, produces (statistically-significant) higher-quality worker outputs compared to the existing static workflow, under equal monetary assumptions.                     
                                                                                    
I will also elaborate the mathematical model underneath the planner -- Markov decision processes (MDPs), and briefly introduce a couple of new approaches that solve MDPs optimally: one that significantly speeds up the convergence of planning by using the problems' graphical information, the other that exploits the availability of external memory to solve much larger problems than previously attempted.                     
                                                                                                                                                              
                                                                                    
Mr. Dai is a Ph.D. candidate at the Department of Computer Science,                  
University of Washington.  He received his MS in CS from the Department of Computer Science, University of  Kentucky.