Loughborough University
Browse

A budget-limited mechanism for category-aware crowdsourcing systems

Download (1.26 MB)
conference contribution
posted on 2021-11-04, 14:02 authored by Yuan Luo, Nick JenningsNick Jennings
Crowdsourcing harnesses human effort to solve computer-hard problems. Such tasks often have different levels of difficulty and workers have varying levels of skill at completing them. With a limited budget, it is important to wisely allocate the spend among the tasks and workers such that the overall outcome is as good as possible. Most existing work addresses this budget allocation problem by assuming that workers have a single level of ability for all tasks. However, this neglects the fact that tasks can belong to a variety of diverse categories and workers may have varying abilities across them. To incorporating such category-awareness, we model the interaction between the crowdsource campaign initiator and the workers as a procurement auction and propose a computationally efficient mechanism, INCARE, to achieve high-quality outcomes given a limited budget. We prove that INCARE is budget feasible, incentive compatible and individually rational. Finally, our experiments on a standard real-world data set show that, compared to the state of the art, INCARE: (i) can improve the accuracy by up to 40%, given a limited budget; and (ii) is significantly more robust to inaccuracies in prior information about each task's difficulty.

History

Published in

Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

Volume

2020-May

Pages

780 - 788

Source

International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020

Publisher

IFAAMAS

Version

  • VoR (Version of Record)

Rights holder

© 2020 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Publisher statement

Permission to make digital or hard copies of portions of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.

Acceptance date

2020-05-01

Publication date

2020-05-05

Copyright date

2020

ISBN

9781450375184

ISSN

1548-8403

eISSN

1558-2914

Language

  • en

Location

Auckland, New Zealand

Event dates

May 9–13, 2020

Depositor

Deposit date: 4 November 2021

Usage metrics

    Loughborough Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC