PROJECT TITLE :

Majority Voting and Pairing with Multiple Noisy Labeling - 2017

ABSTRACT:

With the crowdsourcing of small tasks becoming easier, it's doable to obtain non-professional/imperfect labels at low price. With low-price imperfect labeling, it is simple to collect multiple labels for the same knowledge things. This paper proposes ways of utilizing these multiple labels for supervised learning, based mostly on 2 basic concepts: majority voting and pairing. We show several interesting results primarily based on our experiments. (i) The methods based on the bulk voting plan work well below the case where the knowledge level is high. (ii) On the contrary, the pairing ways are a lot of preferable below true where the certainty level is low. (iii) Among the bulk voting methods, soft majority voting can scale back the bias and roughness, and perform higher than majority voting. (iv) Pairing can utterly avoid the bias by having each sides (potentially correct and incorrect/noisy information) thought-about. Beta estimation is applied to scale back the impact of the noise in pairing. Our experimental results show that pairing with Beta estimation forever performs well beneath different certainty levels. (v) All strategies investigated are labeling quality agnostic methods for real-world applications, and a number of them perform higher than or a minimum of very shut to the gnostic strategies.