Seminar – Selection regimes and selection errors

How can the selection of innovation projects be designed to reduce false positives (investments that shouldn’t have been made) and false negatives (investments that should have been made but weren’t)? Prior research has provided important theoretical insights while largely treating the set of alternatives available for selection as given. While this assumption has been relaxed in recent studies of how selection regimes affect idea generation and selection within an organization, ideas are increasingly generated outside organizational boundaries by innovators who can take their ideas to multiple organizations for evaluation. Our understanding of selection errors in real organizations facing these issues has been limited due to difficulties in collecting decision and outcome data on a complete set of proposed projects. We combine qualitative and quantitative data on a mobile application accelerator to understand how it implemented three different selection regimes over time. Using data on all 3,580 submissions to the accelerator, we collected outcomes for all funded and rejected projects to measure false positives and negatives before empirically evaluating the three selection regimes’ effectiveness in avoiding selection errors. Our findings show that selection regime changes affect both the pool of projects submitted for selection and the errors made in selecting among them. In the last regime that increased submission quality through greater emphasis on applicant track record and additional layers of screening, evaluators were more likely to make false positive and negative decisions. Additional analyses suggest mean reversion combined with over-weighting of applicant track record and adverse selection as likely mechanisms.

Room T09-67, Mandeville Building