It is well known that the high cost of installing renewable energy systems (RES) is still a barrier to overcome in the promotion of net-zero energy buildings (NZEB). The current practice in selecting the optimal RES system for NZEB basically involves a consideration of all possible design options, which is usually not a cost-effective process when compared with costs for no or for less RES in buildings. This study, therefore, introduces a penalty cost in the RES design process for NZEB, which aims to ensure that the cost-effective design option is the system which provides for a net-zero/positive energy building. The proposed penalty cost function is investigated in three scenarios (two scenarios with penalty costs and one scenario without a penalty cost) based on the Hong Kong Zero Carbon Building. It is found that the total cost of the building under a safety factor of 0.0 is increased by about 70% when a penalty cost is considered whilst it is reduced by 39% (for penalty cost 1) and 48.9% (for penalty cost 2) for the building under a safety factor of 1.0 (i.e., NZEB), respectively. In addition, three fitting formulas were derived for designers to better understand the relationship between the total cost and safety factor. The idea of developing a penalty cost mechanism provides a progressive perspective to assist the promotion of NZEB in terms of both governments as well as building designers/owners.