Project

Back to overview

Dialectical bootstrapping:?A new paradigm to improve individual judgment

English title Dialectical bootstrapping:?A new paradigm to improve individual judgment
Applicant Herzog Stefan
Number 129572
Funding scheme Project funding (Div. I-III)
Research institution Cognitive and Decision Sciences Fakultät für Psychologie Universität Basel
Institution of higher education University of Basel - BS
Main discipline Psychology
Start/End 01.01.2011 - 28.02.2017
Approved amount 270'925.00
Show all

Keywords (12)

The Wisdom of Crowds; Dialectical bootstrapping; Judgmental accuracy; Forecast combination; Cognitive tools; Psychological engineering; Cognitive psychology; Prediction; Debiasing; judgments under uncertainty; judgment aggregation; advice taking

Lay Summary (English)

Lead
Lay summary
We proposed "dialectical bootstrapping"-simulating the "wisdom of crowds" within a single mind-as technique to improve individual judgment (Herzog & Hertwig, 2009). This project tests the robustness of dialectical bootstrapping and whether people can be taught to use it.BACKGROUNDDifferent lines of research have addressed how to best make quantitative predictions, including psychology, management science, computer science, statistics and medicine. One time proven way to improve judgment is the following: When there are several different plausible predictions (stemming from different experts and/or different statistical procedures) and no reliable track record about their past performance are available, average those predictions. Averages of predictions outperform the typical predictions in the set and can even outperform the best single prediction. Can a single person benefit from averaging without actually consulting other people? We proposed a novel approach to improve individual judgment called "dialectical bootstrapping", which enables different opinions to be generated and combined by the same person, thus simulating the "wisdom of crowds" within a single mind (Herzog & Hertwig, 2009).GOALWe have two main goals: First, we test the robustness and boundary conditions of dialectical bootstrapping by investigating its effectiveness in different domains and using different procedures. Second, we examine several psychological aspects of dialectical bootstrapping, namely whether people spontaneously use dialectical bootstrapping, whether the tool can be taught to people and whether people are prepared to combine conflicting estimates at all when the conflict's source is their own mind.RELEVANCEDialectical bootstrapping promises to be a practical tool to improve quantitative judgments. In many settings, as for example in finance, medical and managerial decision making, there are successful decision aids available that can be applied in routine decision making situations. However, whenever new situations emerge, decision makers often do not have the time, resources or data to construct appropriate statistical models or to seek advice from other people. Instead, they could try to tap into the wisdom of the crowd in their own mind by applying dialectical bootstrapping.---Herzog, S. M., & Hertwig, R. (2009). The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psychological Science, 20, 231-237. doi:10.1111/j.1467-9280.2009.02271.xPrincipal investigator: Dr. Stefan Herzog, Cognitive and Decision Sciences, Department of Psychology, University of Basel, Basel, Switzerland http:www.psycho.unibas.ch/herz
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
Strategy Selection Versus Strategy Blending: A Predictive Perspective on Single- and Multi-Strategy Accounts in Multiple-Cue EstimationStrategy Selection vs. Blending
Herzog Stefan M., von Helversen Bettina (2016), Strategy Selection Versus Strategy Blending: A Predictive Perspective on Single- and Multi-Strategy Accounts in Multiple-Cue EstimationStrategy Selection vs. Blending, in Journal of Behavioral Decision Making, -.
Harnessing the wisdom of the inner crowd
Herzog Stefan M., Hertwig Ralph (2014), Harnessing the wisdom of the inner crowd, in Trends in Cognitive Sciences, 18(10), 504-506.
Think twice and then: Combining or choosing in dialectical bootstrapping?
Herzog Stefan M., Hertwig Ralph (2014), Think twice and then: Combining or choosing in dialectical bootstrapping?, in Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(1), 218-232.
Blending and choosing within one mind: Should judgments be based on exemplars, rules, or both?
Herzog Stefan M, von Helversen Bettina (2013), Blending and choosing within one mind: Should judgments be based on exemplars, rules, or both?, in Cooperative minds: Social interaction and group dynamics. Proceedings of the 35th Annual Conference , 2536-2541, Cognitive Science Society, Austin, TX2536-2541.
The crowd within and the benefits of dialectical bootstrapping: a reply to white and antonakis (2013).
Herzog Stefan M, Hertwig Ralph (2013), The crowd within and the benefits of dialectical bootstrapping: a reply to white and antonakis (2013)., in Psychological science, 24(1), 117-9.
The wisdom of ignorant crowds: Predicting sport outcomes by mere recognition
Herzog SM, Hertwig R (2011), The wisdom of ignorant crowds: Predicting sport outcomes by mere recognition, in JUDGMENT AND DECISION MAKING, 6(1), 58-72.

Collaboration

Group / person Country
Types of collaboration
Bettina von Helversen, Prof. Dr., Universität Zürich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
37th Annual Meeting of the Society for Judgment and Decisions Making Poster Harnessing the wisdom of the inner crowd by exploiting the confidence in your decisions 18.11.2016 Boston, United States of America Hertwig Ralph; Litvinova Aleksandra; Herzog Stefan;
57th Annual Meeting of the Psychonomics Society Poster Harnessing the wisdom of the inner crowd by exploiting the confidence in your decisions 17.11.2016 Boston, United States of America Litvinova Aleksandra; Hertwig Ralph; Herzog Stefan;
Talk presented at the JDMx 2016 Meeting, Fakultät fuer Psychologie, Universität Basel Talk given at a conference Harnessing the wisdom of the inner crowd by exploiting the confidence in your decisions 09.06.2016 Basel, Switzerland Litvinova Aleksandra; Hertwig Ralph; Herzog Stefan;
25th Subjective Probability, Utility, and Decision Making (SPUDM) Conference Talk given at a conference Improving the wisdom of the inner crowd, with confidence 16.08.2015 Budapest, Hungary Herzog Stefan; Phillips Nathaniel David; Hertwig Ralph;
Management Seminar series at the Department of Economics and Business, Universitat Pompeu Fabra Individual talk Harnessing the wisdom of the inner crowd 19.06.2014 Barcelona, Spain Herzog Stefan;
35th Annual Conference of the Cognitive Science Society Poster Blending and choosing within one mind: Should judgments be based on exemplars, rules or both? 31.07.2013 Berlin, Germany Herzog Stefan;
Wisdom of the Crowd Conference Individual talk The wisdom of crowds within a single mind 18.04.2013 Irvine, USA, United States of America Herzog Stefan;
55. Tagung experimentell arbeitender Psychologen Talk given at a conference Modeling Bayesian inference judgments 24.03.2013 Wien, Austria Hertwig Ralph; Phillips Nathaniel David; Herzog Stefan;
33rd Annual Meeting of the Society for Judgment and Decision Making Talk given at a conference Contradicting yourself makes you more Bayesian: Averaging non-Bayesian judgments with dialectical bootstrapping improves judgments 16.11.2012 Minneapolis, United States of America Phillips Nathaniel David; Herzog Stefan; Hertwig Ralph;
54th Tagung experimentell arbeitender Psychologen Talk given at a conference How groups can act (more) Bayesian without having any Bayesians 01.04.2012 Mannheim, Germany Herzog Stefan; Phillips Nathaniel David; Hertwig Ralph;
54th Tagung experimentell arbeitender Psychologen Talk given at a conference Averaging analytical and intuitive judgment with dialectical bootstrapping: The wisdom of an intuitive–analytical crowd within one mind 01.04.2012 Mannheim, Deutschland, Germany Hertwig Ralph; Herzog Stefan;
32nd Annual Meeting of the Society for Judgment and Decision Making Poster Averaging multiple intuitive strategies improves performance in Bayesian estimation tasks: How groups can act (more) Bayesian without being Bayesian 04.11.2011 Seattle, United States of America Phillips Nathaniel David; Hertwig Ralph; Herzog Stefan;
2011 32nd Annual Meeting of the Society for Judgment and Decision Making Talk given at a conference Averaging analytical and intuitive judgment with dialectical bootstrapping: The wisdom of an intuitive analytical crowd within one mind 04.11.2011 Seattle, USA, United States of America Herzog Stefan; Hertwig Ralph;
2011 International Small-Group Meeting, „What is adaptive cognition and behavior?“ Talk given at a conference The wisdom of crowds within one mind: The benefits of blending opinions and cognitive processes 08.09.2011 Heidelberg, Germany, Germany Herzog Stefan;
4th Workshop for Judgment and Decision making Talk given at a conference How to improve Bayesian inferences by contradicting oneself: Dialectical bootstrapping 03.08.2011 Bonn, Germany, Germany Herzog Stefan; Hertwig Ralph; Phillips Nathaniel David;
10th Annual Summer Interdisciplinary Conference (ASIC) Talk given at a conference The benefits of blending cognitive processes within one mind 07.07.2011 Caldes de Boi, Pyrenees, Spain, Spain Herzog Stefan;
Social and Economic Psychology Colloquium, University of Basel Individual talk The "wisdom of crowds" in one mind: Dialectical bootstrapping and the benefits of blending different cognitive processes 28.04.2011 Basel, Switzerland, Switzerland Herzog Stefan;
53rd Tagung experimentell arbeitender Psychologen Talk given at a conference The benefits of blending cognitive processes within one mind 13.03.2011 Halle, Germany, Germany Herzog Stefan;


Abstract

Experts and laypeople alike cannot help but make judgments under uncertainty in a wide range of situations. Different lines of research have addressed how to best make quantitative predictions, including psychology, management science, computer science, statistics and medicine. One time proven way to improve judgment is the following: When there are several different plausible predictions (stemming from different experts and/or different statistical procedures) and no reliable track record about their past performance, mechanically average those predictions. Averages of predictions outperform the typical predictions in the set and can even outperform the best single prediction. Averaging estimates increases accuracy in two ways: It cancels out random error, and it can reduce systematic error. The key insight is that averaging succeeds in decreasing error to the extent that the individual errors are non-redundant. Models of forecast combination demonstrate that averaging is a good default strategy and should not be abandoned without good reasons.In stark contrast to the usefulness of averaging, most people are reluctant to use the power of averaging to improve their judgment, mostly because they fail to appreciate the averaging principle. They do not consider that errors of opposing sign cancel each other out. Instead they tend to assume that the accuracy of the average is equal to the average accuracy of the sources considered. As a consequence, people often try, for instance, to pick the best advisor from a set of advisors-with typically only modest success, thereby forgoing the gain they could have achieved had they averaged rather than selected what they thought is the best advice.Aggregation, however, requires more than one prediction. What can a person do if he or she is unable to consult a crowd of people or models? Can a single person still benefit from the wisdom of the crowd without actually consulting other people? That is, can cognitive diversity within one person give rise to “The Wisdom of Crowds” in a single mind? This research project investigates a novel approach to improve individual judgment that was first proposed by Herzog and Hertwig (2009)-a mental tool called “dialectical bootstrapping”, which enables different opinions to be generated and combined by the same person, thus simulating the wisdom of crowds within a single mind. According to this framework, an individual can reduce her overall error by averaging her first estimate with a second, plausible estimate (termed “dialectical estimate”), which is likely to have a different error to the first estimate. Herzog and Hertwig provided an existence proof for dialectical bootstrapping.This research proposal aims to extend our research on dialectical bootstrapping in two ways: First, we test the robustness and boundary conditions of dialectical bootstrapping by investigating its effectiveness in different domains, using different elicitation procedures, and assessing the marginal benefits of additional dialectical estimates. Second, we examine several psychological aspects of dialectical bootstrapping, namely whether people spontaneously use dialectical bootstrapping, whether the tool can be taught to people and whether people are prepared to combine conflicting estimates at all when the conflict’s source is their own mind.
-