There is currently a great interest to combine electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) to study brain function. Earlier studies have shown different EEG components to correlate well with the fMRI signal arguing for a complex relationship between both measurements. In this study, using separate EEG and fMRI measurements, we show that (1) 0.1 ms visual stimulation evokes detectable hemodynamic and visual-evoked potential (VEP) responses, (2) the negative VEP deflection at approximately 80 ms (N2) co-varies with stimulus duration/intensity such as with blood oxygenation level-dependent (BOLD) response; the positive deflection at approximately 120 ms (P2) does not, and (3) although the N2 VEP-BOLD relationship is approximately linear, deviation is evident at the limit of zero N2 VEP. The latter finding argues that, although EEG and fMRI measurements can co-vary, they reflect partially independent processes in the brain tissue. Finally, it is shown that the stimulus-induced impulse response function (IRF) at 0.1 ms and the intrinsic IRF during rest have different temporal dynamics, possibly due to predominance of neuromodulation during rest as compared with neurotransmission during stimulation. These results extend earlier findings regarding VEP-BOLD coupling and highlight the component- and context-dependency of the relationship between evoked potentials and hemodynamic responses.