Silicon is of significant interest as a next-generation anode material for lithium-ion batteries due to its extremely high capacity. The reaction of lithium with crystalline silicon is known to present a rich range of phenomena, including electrochemical solid state amorphization, crystallization at full lithiation of a Li15Si4 phase, hysteresis in the first lithiation-delithiation cycle, and highly anisotropic lithiation in crystalline samples. Very little is known about these processes at an atomistic level, however. To provide fundamental insights into these issues, we develop and apply a first principles, history-dependent, lithium insertion and removal algorithm to model the process of lithiation and subsequent delithiation of crystalline Si. The simulations give a realistic atomistic picture of lithiation demonstrating, for the first time, the amorphization process and hinting at the formation of the Li15Si4 phase. Voltages obtained from the simulations show that lithiation of the (110) surface is thermodynamically more favorable than lithiation of the (100) or (111) surfaces, providing an explanation for the drastic lithiation anisotropy seen in experiments on Si micro- and nanostructures. Analysis of the delithiation and relithiation processes also provides insights into the underlying physics of the lithiation-delithiation hysteresis, thus providing firm conceptual foundations for future design of improved Si-based anodes for Li ion battery applications.