An anisotropic carbon nanotube (CNT)-polymer composite for bolometric applications in the mid-IR spectral range (2.520 mu m) is studied. Composite alignment in conjunction with non-uniform distribution of CNTs in the polymer matrix allows for a significant enhancement of the temperature coefficient of resistance (0.82% K-1) with respect to uniform composite (0.24% K-1). As a result a responsivity of approximate to 500 V W-1 is reached, which is the highest for CNT-based bolometers reported to date. Such remarkable optical and thermal characteristics are explained in terms of fluctuation tunneling theory taking into account the composite anisotropy and the gradient of the CNT concentration. Flatness of the photoresponse in the broad spectral mid-IR range and enhanced responsivity provide a great potential for the use of such novel composite for applications in IR spectroscopy and thermal imaging.