With complex dynamic nature, Heat Exchanger Networks (HENs) should be operated successfully throughout the whole time horizon even facing the stochastic and the time-varying disturbances. In current studies, overdesigning HENs is a commonly adopted strategy to deal with the stochastic disturbances and also the flexible design. However, it is not a good choice to find the tradeoff between the dynamic flexibility and the total annual cost of HENs. In this paper, a new optimization-based framework for designing dynamic flexible HENs is presented. The key idea is to consider the ranges of variations in stream output temperatures to explore such tradeoff. This allows a HEN to work under the stochastic and the time-varying disturbances without losing stream temperature targets while keeping the economically optimal energy integration. This work begins with the multiple disturbances, and then dynamic flexibility analysis is employed to determine the Generalized Critical Operating Points (GCOPs) that are proposed to indicate the bottleneck of dynamic flexibility. As for each GCOP, the HEN retrofit is performed for the capability of accommodating the stochastic and the time-varying disturbances. These are formulated as a superstructure-based Mixed Integer Nonlinear Programming model with the objective of minimizing the total annual cost. Three cases are given to demonstrate the application of the proposed framework. Dynamic simulation and quantitative measures show the overall economic performance and the capability of accommodating the multiple disturbances.