Background: We compared high-resolution magnetic resonance imaging (MRI) with computed tomography (CT) in the assessment of tumor infiltration in surrounding structures for locally advanced primary and recurrent rectal cancer. Methods: Twenty-six patients with operable, locally advanced rectal cancer (15 recurrent and 11 primary) were evaluated with conventional pelvic CT and 1.5-T high-resolution MRI with a quadrature phased-array coil. The images were scored for invasion of nine neighboring pelvic structures, and the results were compared with surgical and histologic findings. Results: A total of 234 structures in 26 patients was evaluated for tumor invasion. For MRI the, sensitivity was 97% and the specificity 98%; for CT, the sensitivity was 70% and the specificity was 85%. The difference in performance was statistically significant (p <0.001). The failure most frequently made on CT was the false-positive prediction of pelvic floor and piriform muscle invasion (14), whereas MRI showed only four false-positive predictions. MRI correctly predicted all four cases of sacral bone invasion, three of which were missed by CT. MRI was accurate in 20 patients (80%) and CT in only five patients (19%). Conclusion: High-resolution MRI using a quadrature phased-array coil is highly accurate and superior to CT in predicting tumor infiltration in surrounding structures for locally advanced primary or recurrent rectal cancer and is recommended in the preoperative work-up of these tumors.