In this cross-sectional study, we have sought to examine the additive vs synergistic contribution of beta-amyloid (Aβ) and white matter hyperintensities (WMH) across seven cognitive domains in 104 cognitively normal elderly individuals. A primary objective was to measure the extent to which age-related differences in cognition are driven by measurable brain pathology. All participants underwent neuropsychological assessment along with PET-MRI imaging using 11C-Pittsburgh Compound B for Aβ quantification. WMH severity was quantified using the ARWMC scale. Stepwise regressions, moderation and mediation modelling were performed. Our findings indicate that Aβ and WMH contribute additively to poorer performance in working memory and language while they carry synergistic associations with executive functions and attention. We also found that Aβ deposition single-handedly predicted poorer episodic memory performance. Most importantly, we demonstrated through mediation modelling that age’s influence over episodic memory, working memory, executive functions and language is fully mediated by brain pathology. This study permits to conclude that, in the healthy elderly; 1) Aβ burden and WMH have synergistic associations with cognition and; 2) age-related differences in cognitive performance are for the most part driven by brain pathology associated with dementia.