In the paper, we investigate the exponential stability of nonlinear delayed systems with destabilizing and stabilizing delayed impulses, respectively. Specifically, the study can be divided into two cases: (1) stability of delayed systems with destabilizing delayed impulses, where the time delays in impulses can be flexible and even larger than the length of impulsive interval, and (2) stability of delayed systems with stabilizing delayed impulses, where the time delays in impulses are flexible between two consecutive impulsive instants. In order to address the time-delay term in impulses, the concept of average impulsive delay (AID) is proposed. Using the ideas of average impulsive interval and AID, we present some Lyapunov-based exponential stability criteria for delayed systems with average-delay impulses, where the delays in impulses satisfy the proposed AID condition. It is shown that time delay in impulse has double effects, namely, it may destabilize a stable system or stabilize an unstable system. Interestingly, it is also shown that for some stable delayed systems with stabilizing delayed impulses, under certain conditions, the stability can be ensured regardless of the size of delay in continuous dynamics. Further, we apply the theoretical results to the impulsive synchronization control of Chua's circuits with both transmission delay and sampling delay. Finally, some examples are given to illustrate the validity of the derived results.