Complex feedback systems are ubiquitous in biology. Modeling such systems with mass action laws or master equations requires information rarely measured directly. Thus rates and reaction topologies are often treated as adjustable parameters. Here we present a general stochastic modeling method for small chemical and biochemical systems with emphasis on feedback systems. The method, Maximum Caliber (MaxCal), is more parsimonious than others in constructing dynamical models requiring fewer model assumptions and parameters to capture the effects of feedback. MaxCal is the dynamical analogue of Maximum Entropy. It uses average rate quantities and correlations obtained from short experimental trajectories to construct dynamical models. We illustrate the method on the bistable genetic toggle switch. To test our method, we generate synthetic data from an underlying stochastic model. MaxCal reliably infers the statistics of the stochastic bistability and other full dynamical distributions of the simulated data, without having to invoke complex reaction schemes. The method should be broadly applicable to other systems.