The impact of the geometric phase on the time evolution of quantum-mechanical wave packets is studied theoretically. Two model systems of coupled electronic potential energy surfaces are compared. One of them, the well-known EXe Jahn-Teller system, comprises two conically intersecting surfaces, and the dynamics is subject to the geometric phase. The other system, describing the (E+A)Xe Pseudo-Jahn-Teller effect, comprises three intersecting surfaces and the dynamics is not subject to a geometric phase. Apart from the geometric phase, the coupling to the upper surface is verified to be negligible for low-energy wave packet motion. Still, the geometric phase leads to a pronounced difference of low-energy wave packet dynamics in both systems. Most significant is the phenomenon of destructive self-interference of the two parts of the wave packet that encircle the conical intersection on opposite sides. The importance of the resulting different shape of the wave packet for a fs pump-probe spectrum is pointed out.