We discuss the relation of a certain type of generalized Lyapunov equations to Gramians of stochastic and bilinear systems together with the corresponding energy functionals. While Gramians and energy functionals of stochastic linear systems show a strong correspondence to the analogous objects for deterministic linear systems, the relation of Gramians and energy functionals for bilinear systems is less obvious. We discuss results from the literature for the latter problem and provide new characterizations of input and output energies of bilinear systems in terms of algebraic Gramians satisfying generalized Lyapunov equations. In any of the considered cases, the definition of algebraic Gramians allows us to compute balancing transformations and implies model reduction methods analogous to balanced truncation for linear deterministic systems. We illustrate the performance of these model reduction methods by showing numerical experiments for different bilinear systems.