We consider binary polymer blends where the energetic interactions between the three different pairs of monomers are not equal to each other. From computer simulations in the isothermal-isobaric ensemble, we show that these disparate energetic interactions yield different packing behavior even at the level of the pure components. This results in the pure materials having unequal densities and cohesive energies. In the context of mixtures we have considered two cases. In the system termed "attractive" we have employed Lennard-Jones interactions which favor the formation of dissimilar 1-2 contacts. The residual chemical potential changes on mixing for both components, independently, appear to vary parabolically with system composition. If one chose to model these results with the Flory model [Or alternately, the polymer analog of the regular solution theory], then the chemical potential changes on mixing for each component would appear to be describable by a composition independent value of the chi parameter. However, the effective chi values derived from the two components are not equivalent. Since this finding violates thermodynamic consistency, it implies that the effective single chi parameter characterizing the free energy of mixing must be composition dependent. Similar results were obtained in the case of a "nominally athermal" blend, where the 1-2 interaction energy was selected to yield chi = 0 if the blend followed the Flory assumptions of incompressibility and randomly mixed chain segments. We show that these results can be explained on the basis of energetic effects and nonrandom mixing which is triggered by differences in the packing behavior of the pure components. Finally, this packing disparity also causes polymer blends with nominal chi = 0 in terms of the Flory lattice definition to phase separate. This phenomenon is a consequence of the coupling between packing and energetic interactions which yields an unfavorable enthalpy change on mixing. We conclude by emphasizing, in agreement with past theoretical calculations, the importance of packing effects in describing the thermodynamics of polymer systems, an issue which must be included if an improved description of polymer thermodynamics is to be achieved.