In this report, we employ surface-initiated atom transfer radical polymerization (SI-ATRP) to graft a thermoresponsive polymer, poly(N-isopropylacrylamide) (PNIPAM), of controlled thickness from porous silicon (pSi) films to produce a stimulus-responsive inorganic organic composite material. The optical properties of this material are studied using interferometric reflectance spectroscopy (IRS) above and below the lower critical solution temperature (LCST) of the PNIPAM graft polymer with regard to variation of pore sizes and thickness of the pSi layer (using discrete samples and pSi gradients) and also the thickness of the PNIPAM coatings. Our investigations of the composite's thermal switching properties show that pore size, pSi layer thickness, and PNIPAM coating thickness critically influence the material's thermoresponsiveness. This composite material has considerable potential for a range of applications including temperature sensors and feedback controlled drug release. Indeed, we demonstrate that modulation of the temperature around the LCST significantly alters the rate of release of the fluorescent anticancer drug camptothecin from the pSi-PNIPAM composite films.