Relationships between the Jaynes and Shannon information entropies, both of which are related to the correlation energy, are given and the physical implications are explored. Evidence is provided from a study of the Be and Ne isoelectronic sequences that the Shannon entropy is more sensitive to the effects of the nuclear charge than is the Jaynes entropy. It is shown that the Jaynes entropy may be considered as the difference between the mean orbital Shannon entropy per electron and the mean orbital Shannon entropy of one electron. These mean orbital entropies display the same functional behavior as the total Shannon entropy throughout the isoelectronic sequence. We demonstrate that the Jaynes entropy may be considered as the sum of the Kullback-Leibler distance entropies occurring between natural spin orbital densities and a reference point whose occupation number is one. The large Z asymptotic behavior of the Jaynes and Shannon entropies is discussed. From a study of a Ne isonuclear sequence, it is shown that both Shannon and Jaynes entropies increase with the number of electrons, illustrating that these quantities are sensitive to the amount of electronic interactions present in the system. Relationships for the changes in the Shannon and Jaynes entropies with respect to the number of electrons for an isonuclear sequence are obtained, which depend on the electron affinity, first and second ionization potentials, and the hardness and chemical potential of the system.