Difficulty interpreting facial expressions is an important component of social deficits seen in Autism Spectrum Disorders (ASDs). The amygdalo-fusiform white matter tract is theorized to contribute to facial expression interpretation. The current study examined relationships between white matter integrity (WMI) along the aymgdalo-fusiform pathway, facial emotion perception (FEP), and socialization skill (SS) in a sample of typically developing (TD) participants and participants with ASDs. Groups were differentiated by WMI along the amygdalo-fusiform tract. Results suggested unexpected negative relationships between WMI and SS, and between WMI and FEP in the ASD group. A group interaction was suggested for WMI and FEP such that participants with ASDs demonstrated a negative relationship, whereas TD participants displayed the hypothesized positive relationship. Overall, findings suggest that measures of WMI along the amygdalo-fusiform tract should not be conceptualized identically in participants with ASDs and controls.