The effect of hyperbolic modes on near-field thermophotovoltaic (TPV) system performance is investigated by implementing a hyperbolic metamaterial (HMM) radiator. Specifically, the near-field TPV system consists of a 2D grating tungsten radiator and a gallium antimonide (GaSb) cell separated by a gap thickness varying between 100 nm and 500 nm. The temperatures of the radiator and the TPV cell are fixed at 2000 K and 300 K, respectively. The effective medium theory (EMT) is proposed to solve the near-field radiative heat transfer problem with a uniaxial anisotropic radiator. Near-field TPV performance is evaluated via maximum power output and conversion efficiency. Enhanced radiative heat transfer is observed within the spectral band where hyperbolic modes are supported by the radiator. As a result, the radiative heat flux absorbed by the cell is enhanced from broadband tunneling of evanescent waves. Furthermore, near-field TPV maximum power output and conversion efficiency are improved. The highest maximum power output and conversion efficiency are 4.28 x 10(5) W m(-2) and 35%, respectively. When the gap thickness is smaller than 300 nm, the effect of hyperbolic modes is strong, and radiative losses affect the conversion efficiency as radiative flux is enhanced both below and above the cell band gap. For gap thicknesses larger than 300 nm, the effect of hyperbolic modes is weak and the conversion efficiency is mostly affected by the spatial distribution of radiative flux absorbed in the cell. In this study, radiative heat flux enhancement and the improvement of near-field TPV performance are attributed to hyperbolic modes supported by the radiator. These findings will further contribute to the design of near-field TPV experimental systems outperforming their far-field counterparts. (C) 2017 Elsevier Ltd. All rights reserved.