Ulric B. and Evelyn L. Bray Seminar

There is increasing concern about reproducibility in science. Factors contributing to a lack of reproducibility include low statistical power, the testing of hypotheses with low prior probability of being true, and publication bias. In a number of projects we are testing whether prediction markets can be used as a tool to assess the reproducibility of published scientific results. We compare prediction market forecasts to surveys of market participant's individual forecasts. The prediction market also allows us to estimate probabilities for the hypotheses to be true. We show that such probabilities can be assigned to a hypothesis for different points in the testing process: before and after the replication has been performed, and even before the original study has been performed. We discuss in detail our first prediction market on the reproducibility of psychology studies from prominent psychology journals, which is part of the Open Science Collaboration Reproducibility Project.