In childhood frontal lobe epilepsy (FLE), cognitive impairment and educational underachievement are serious, well-known co-morbidities. The broad scale of affected cognitive domains suggests wide-spread network disturbances that not only involves, but also extends beyond the frontal lobe. In this study we have investigated whole brain connectional properties of children with FLE in relation to their cognitive impairment and compared them with healthy controls. Functional connectivity (FC) of the networks was derived from dynamic fluctuations of resting state fMRI and structural connectivity (SC) was obtained from fiber tractograms of diffusion weighted MRI. The whole brain network was characterized with graph theoretical metrics and decomposed into modules. Subsequently, the graph metrics and the connectivity within and between modules were related to cognitive performance. Functional network disturbances in FLE were related to increased clustering, increased path length, and stronger modularity compared to healthy controls, which was accompanied by stronger within- and weaker between-module functional connectivity. Although structural path length and clustering appeared normal in children with FLE, structural modularity increased with stronger cognitive impairment. It is concluded that decreased coupling between large-scale functional network modules is a hallmark for impaired cognition in childhood FLE.