Clinical and experimental data have implicated the posterior superior temporal gyrus as an important cortical region in the processing of socially relevant stimuli such as gaze following, eye direction, and head orientation. Gaze following and responding to different socio-communicative signals is an important and highly adaptive skill in primates, including humans. Here, we examined whether individual differences in responding to socio- communicative cues was associated with variation in either gray matter (GM) volume and asymmetry in a sample of chimpanzees. Magnetic resonance image scans and behavioral data on receptive joint attention (RJA) was obtained from a sample of 191 chimpanzees. We found that chimpanzees that performed poorly on the RJA task had less GM in the right compared to left hemisphere in the posterior but not anterior superior temporal gyrus. We further found that middle-aged and elderly chimpanzee performed more poorly on the RJA task and had significantly less GM than young-adult and sub-adult chimpanzees. The results are consistent with previous studies implicating the posterior temporal gyrus in the processing of socially relevant information.