Research suggests that intuitions about thought experiments are vulnerable to a wide array of seemingly irrelevant factors. I argue that when arguments hinge on the use of intuitions about thought experiments, research on the subtle factors that affect intuitions must be taken seriously. To demonstrate how failing to consider such psychological influences can undermine an argument, I discuss Pereboom’s four-case manipulation argument. I argue that by failing to consider the impact of subtle psychological influences such as order effects, Pereboom likely mis-identifies what really leads us to have the intuitions that we have about his cases, and this in turn undermines his argument for incompatibilism. Last, I consider objections and discuss how to empirically test my hypothesis.