Voxel-based morphometry (VBM) was used to examine the relation between age and gray matter density cross-sectionally and to study the association between gray matter density and longitudinal decline in performance on cognitive tests in healthy, non-demented elderly individuals. Participants were neuropsychologically tested at baseline and again after 3 years. Thirty-seven subjects (mean age 72.5 years) who showed a decline in cognitive test performance at follow-up were compared with 38 individually matched control subjects (mean age 71.8 years) whose performance did not change over time. Magnetic resonance imaging scans were acquired at follow-up and individual differences in regional gray matter density were examined with VBM. The largest age effects were found in various regions in the prefrontal cortex, the (medial) temporal lobes and the striate cortex. Longitudinal cognitive decline was associated with decreased gray matter density in prefrontal areas, the (medial) temporal lobes and the posterior parietal cortex. These findings suggest that prefrontal and temporal cortical regions are of particular relevance both in aging and age-related cognitive decline in healthy elderly individuals.