In this study, heat sinks in natural convection are thermally optimized using the method of three-dimensional topology optimization. In order to perform three-dimensional topology optimization with low computational cost, a shape-dependent convection model is proposed. This model accounts for the variation of the heat transfer coefficient depending not only on the local shape of the fins but also on the development of the thermal boundary layer. The physical validity of the proposed model is confirmed by the fin geometry of the topology-optimized design that matches the multiscale structures proposed previously by the constructal theory. For further validation, the effective heat transfer coefficient evaluated by the proposed model is compared to that obtained from numerical simulations. Because the new topology-optimized design has a complicated fin geometry, design simplification is performed to yield a more manufacturable design. The thermal performance of the topology-optimized heat sink is compared to that of the radial plate-fin heat sink optimized analytically using an existing correlation. It is found that the topology-optimized heat sink has 13% lower thermal resistance and 48% less mass than the optimized radial plate-fin heat sink. This implies that the three-dimensional topology optimization method suggested in this study can provide a heat sink design with improved thermal performance and reduced mass for various practical applications. (C) 2018 Elsevier Ltd. All rights reserved.