Chromatin regulators play a major role in establishing and maintaining gene expression states. Yet how they control gene expression in single cells, quantitatively and over time, remains unclear. We used time-lapse microscopy to analyze the dynamic effects of four silencers associated with diverse modifications: DNA methylation, histone deacetylation, and histone methylation. For all regulators, silencing and reactivation occurred in all-or-none events, enabling the regulators tomodulate the fraction of cells silenced rather than the amount of gene expression. These dynamics could be described by a three-state model involving stochastic transitions between active, reversibly silent, and irreversibly silent states. Through their individual transition rates, these regulators operate over different time scales and generate distinct types of epigenetic memory. Our results provide a framework for understanding and engineering mammalian chromatin regulation and epigenetic memory.