Rapidly decoding the emotional content of a face is an important skill for successful social behavior. Several Event Related brain Potential (ERP) have indicated that emotional expressions already influence brain activity as early as 100 ms. Some studies hypothesized that this early brain response to fear depends on coarse-magnocellular inputs, which are primarily driven by Low Spatial Frequency (LSF) cues. Until now however, evidence is inconclusive probably due to the divergent methods used to match luminance and contrast across spatial frequencies and emotional stimuli. In the present study, we measured ERPs to LSF and HSF faces with fearful or neutral expressions when contrast and luminance was matched across SF or not. Our findings clearly show that fearful facial expressions increases the amplitude of P1 (only for contrast-luminance equated images) and N170 in comparison to neutral faces but only in LSF faces, irrespective of contrast or luminance equalization, further suggesting that LSF information plays a crucial role in the early brain responses to fear. Furthermore, we found that, irrespective of luminance or contrast equalization, N170 occurred faster when perceiving LSF faces than HSF faces, again emphasizing the primacy of LSF processing in early face perception.