The surface roughness of picosecond laser micromachined sidewalls in silicon is investigated theoretically and experimentally by varying the laser fluence, the focal spot size and the crater overlap. It is shown that the precise spot positioning and the repeated ablation induce a periodic structuring of the sidewall surface. The roughness measured by confocal microscopy along the laser scanning direction (RaX), can be decreased by increasing the crater overlap and the focal spot size. This behavior, confirmed using two different lasers, can be explained by a simple geometrical model. However, above an optimal crater overlap of approximately 80%, the surface roughness increases again. The sources of this limitation are discussed. The roughness measured along the beam propagation direction (RaY) is minimal around a crater overlap of 80% as well. Varying the spot size does not significantly change RaY but impacts on the curvature of the sidewall. Moreover, it is shown that once the sidewall is fully formed, its surface roughness is barely dependent on the number of laser scanning passes.