The dot probe task has become the most widely used experimental paradigm for empirically investigating attention biases in anxiety. However, this task’s classic reaction time index for calculating mean attention bias toward threat has shown very low reliability. In response to these findings, new indices of attention bias variability and trial-level bias score (TL-BS) strategies have been developed. The purpose of the current study is to evaluate the reliability of all dot probe attention bias indices in the scientific literature. Forty-four community participants with social anxiety disorder completed a facial dot probe task at two sessions one week apart. TL-BS indices of mean attention bias and attention bias variability demonstrated superior reliability to other indices. The findings demonstrate that the TL-BS method is psychometrically superior to other indices for measuring mean attention bias to threat and attention bias variability in people with social anxiety disorder.