The crystal and magnetic structures of charge-disproportionated Ca2FeMnO6 were analyzed by neutron powder diffraction. Ca2FeMnO6 is a layered double perovskite oxide with a two-dimensional arrangement of Mn4+ and unusual high valence Fe4+ at room temperature. When cooled, the compound shows charge disproportionation followed by magnetic transition. Around 200 K, the Fe4+ shows the charge disproportionation to Fe3+ and Fe5+, which are ordered in a checkerboard pattern in the two-dimensional FeO6 octahedral layers. The magnetic transition occurs at 95 K, which is much lower than the charge disproportionation temperature. The magnetic structure is commensurate but noncollinear, and the antiferromagnetic coupling of Fe3+ and Fe5+ spins in the FeO6 octahedral layers gives the ferrimagnetic moments. The unique magnetic Structure is described as a result of two-dimensional localization of the ligand holes with effective spins.