Inter-individual variation in energy expenditure is mainly a function of differences in body size and physical activity. Intra-individually, the energy expenditure associated with physical activity, i.e. muscular contractions to perform body postures and -movements, is the most variable component of total energy expenditure. Determinants of activity associated energy expenditure (AEE), as derived from observational and intervention studies are presented. Twin studies showed that most of the between subject variation in AEE is explained by genetic factors. AEE of subjects in the confined environment of a respiration chamber was on average halve the value as observed in the same subjects in free-living conditions with doubly labeled water. In young adults, non-training activity was not affected by exercise training. However, in elderly subjects, exercise training induced an equivalent compensatory decline in non-training activity. Similarly, AEE was reduced during energy restriction and in patients with chronic disease increasing resting energy expenditure. Studies with exercise training showed the reduction is difficult to overcome.