The present study compared network specificity between task-residual and resting state data types. Task-residual data capitalizes on the remaining variance after the mean task-related signal is removed from the time series. This study also examined how inter- and intrahemispheric connectivity (bilateral homologous regions and regions contained within the same hemisphere, respectively) within the language and attention networks change as a result of age. Task-residual functional connectivity evidenced stronger laterality of the language and attention connections and thus greater network specificity than resting state functional connectivity of the same connections. Using task-residual data may be optimal for characterizing the synchronized fluctuations between regions of discrete networks. Furthermore, alterations in intrahemispheric functional connectivity can be observed as early as middle age within the domain-general attention domain.