Ulric B. and Evelyn L. Bray Social Sciences Seminar

Abstract: We report an experiment designed to measure how (and how well) subjects choose between biased sources of instrumentally valuable information. Subjects choose between two information sources with opposing biases in order to inform their guesses of a binary state. By varying the nature of the bias, we vary whether it is optimal to consult sources biased towards or against subjects' prior beliefs. We find that subjects frequently choose sub-optimal information sources, and that these mistakes can be described by a handful of well-defined decision rules. Most common among these is a conformation-seeking rule that guides subjects to systematically choose information sources that are biased towards their priors. Analysis of post-experiment survey questions suggest that subjects follow these rules intentionally and find them normatively appealing. Combined with incentivized belief data and post-experiment cognitive tests, this suggests that mistakes like confirmation-seeking are driven by fundamental errors in reasoning about the informativeness of biased information sources.