Room-temperature ionic liquids (ILs) composed of organic anions bis(trifluoromethanesulfonyl)imide (TFSI), bis(fluorosulfonyl)imide (FSI), and trifluoromethanesulfonate (OTf) exhibit interesting physical properties and are important for many electrochemical applications. TFSI and FSI form "hydrophobic" ILs, immiscible with water but miscible with many organic solvents and polymers; for computer simulation studies, it is thus essential to develop force fields for these anions that are transferable among this wide variety of chemical environments. In this work, we develop entirely ab initio force fields for the TFSI, FSI, and OTf anions and predict the properties of corresponding 1-butyl-3-methylimidazolium ILs. We discuss important subtleties in the force field development related to accurately modeling conformational flexibility, that is, relaxed torsional profiles and intramolecular electrostatic interactions. The TFSI anions have notable conformational flexibility in the IL, and we predict approximately 70% cisoid and 20% transoid conformations, which is largely driven by cation/anion ion-pair interactions and is opposite to the trend expected from the anion ab initio potential energy surface. The favorable interactions between the cation and cisoid TFSI conformations result in a shoulder in the cation/anion radial distribution function at short distances, whereas interconversion between cisoid and transoid conformations occurs on a commensurate time scale as ion diffusion processes. In addition to this physical insight on anion effects, we expect that these force fields will have important applications for studying a variety of complex electrolyte systems.