Ultra-fine grained (UFG) metals fabricated by severe plastic deformation (SPD) sometimes exhibit peculiar mechanical properties. For example, the "hardening by annealing and softening by deformation" was reported in UFG aluminum, which was totally opposite to the behaviors of conventionally coarse-grained materials. In this study, the effect of SPD strain on the peculiar phenomena was investigated. The UFG aluminum was fabricated by various cycles of the accumulative roll-bonding (ARB) process with lubrication at ambient temperature. The specimen ARB-processed by ten cycles certainly showed the peculiar phenomena. On the other hand, the 6-cycle specimen did not show the phenomena but was softened by annealing and hardened by deformation normally. From the results of microstructural characterization, it was suggested that the difference in the change of the mechanical property during annealing and deformation between 6-cycle and 10-cycle specimens was caused by the difference in the grain size and/or the texture components, which depended on the SPD strain.