Dynamic Estimation of Rater Reliability in Regression Tasks using Multi-Armed Bandit Techniques
Document Type Conference Paper
Presented at the Workshop on Machine Learning in Human Computation and Crowdsourcing, in conjunction with ICML 2012
In this paper we show that MAB techniques are suit- able for performing the task of the dynamic estimation of rater reliability. We focus on crowdsourcing scenar- ios where the ratings are numerical values typically in a scale, in contrast to other research in the area which concentrates on binary ratings.