Loughborough University
Browse

Testing the effectiveness of debiasing techniques to reduce overprecision in the elicitation of subjective continuous probability distributions

Download (1.93 MB)
journal contribution
posted on 2022-09-09, 11:25 authored by Valentina Ferretti, Gilberto Montibeller, Detlof von Winterfeldt

Formal expert elicitation is a widely used method for quantifying uncertain variables in decision and risk analysis. When estimating uncertain variables, experts and laypeople exhibit overprecision, meaning that the ranges of their estimates are too narrow. Overprecision, a form of overconfidence, is pervasive and hard to correct, thus posing a challenge to expert elicitation. Following the increasing interest toward improving judgments in Behavioral Operational Research (OR), and the limited evidence about the effectiveness of debiasing tools, the aim of our research is to test the effectiveness of commonly employed practices for debiasing overprecision. We conducted two experiments, testing a set of debiasing techniques when eliciting points of a cumulative distribution functions for general knowledge questions. The debiasing procedures included hypothetical bets, counterfactual argumentation, and automatic stretching to increase the ranges of subjects’ initial estimates. We find that two debiasing strategies that require further reasoning after initial estimates (hypothetical bets and counterfactuals) were not very effective for reducing overprecision, while the use of multipliers that increase the initial range of distributions, coupled with a re-elicitation of the distribution with the new range, provided more positive results. We provide some recommendations for expert elicitation in OR practice, based on our findings, and suggest avenues for further research into debiasing overprecision. 

Funding

Center for Risk and Economic Analysis of Threats and Emergencies, University of Southern California

History

School

  • Business and Economics

Department

  • Business

Published in

European Journal of Operational Research

Volume

304

Issue

2

Pages

661-675

Publisher

Elsevier BV

Version

  • VoR (Version of Record)

Rights holder

© The Author(s)

Publisher statement

This is an Open Access Article. It is published by Elsevier under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Licence (CC BY-NC-ND). Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/

Acceptance date

2022-04-07

Publication date

2022-04-14

Copyright date

2022

ISSN

0377-2217

Language

  • en

Depositor

Gilberto Montibeller. Deposit date: 26 May 2022

Usage metrics

    Loughborough Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC