The Adam-Gibbs view of the glass transition relates the relaxation time to the configurational entropy, which goes continuously to zero at the so-called Kauzmann temperature. We examine this scenario in the context of a dimer model with an entropy-vanishing phase transition and stochastic loop dynamics. We propose a coarse-grained master equation for the order parameter dynamics which is used to compute the time-dependent autocorrelation function and the associated relaxation time. Using a combination of exact results, scaling arguments, and numerical diagonalizations of the master equation, we find nonexponential relaxation and a Vogel-Fulcher divergence of the relaxation time in the vicinity of the phase transition. Since in the dimer model the entropy stays finite all the way to the phase transition point and then jumps discontinuously to zero, we demonstrate a clear departure from the Adam-Gibbs scenario. Dimer coverings are the "inherent structures" of the canonical frustrated system, the triangular Ising antiferromagnet. Therefore, our results provide a new scenario for the glass transition in supercooled liquids in terms of inherent structure dynamics.