Monitoring the operation of a pyrolysis reactor is always challenging due to the extremely high-operating temperature (over 800 degrees C) in the fired furnace. To improve current monitoring capability, a monitoring framework is proposed that builds upon thermal photography to provide a detailed view inside the fired furnace. Based on the infrared images generated from the temperature data provided by cameras, a deep learning approach is introduced to automatically identify tube regions from the raw images. The pixel-wise tube segmentation network is named Res50-UNet, which combines the popular ResNet-50 and U-Net architectures. By this approach, the precise temperature and shape on pyrolysis tubes are monitored. The control limits are eventually drawn by the adaptive k-nearest neighbor method to raise alarms for faults. Through testing over real plant data, the framework assists process operators by providing in-depth operating information of the reactor and fault diagnosis. (c) 2018 American Institute of Chemical Engineers AIChE J, 65: 582-591, 2019