Experiments on choice under risk typically involve multiple decisions by individual subjects. The choice of mechanism for selecting decision(s) for payoff is an essential design feature unless subjects isolate each one of the multiple decisions. We review theoretical properties of mechanisms including properties of two new mechanisms introduced herein. We report an experiment with several payoff mechanisms that generate data that show systematic differences across mechanisms in subjects’ revealed risk preferences. We illustrate the importance of these mechanism effects by identifying their implications for tests of classic properties of theories of decision under risk. We also identify behavioral properties of mechanisms that diverge from theoretical incentive compatibility and may introduce bias in risk preference elicitation.