The green fluorescent protein (GFP) isolated from the jellyfish Aequorea victoria is a very useful reporter for real-time bioprocess sensing. GFP culture fluorescence is a composite signal that can be influenced by factors such as culture autofluorescence, inner filter effect (IFE), and photobleaching. These factors complicate accurate estimation of GFP concentrations from the culture fluorescence. IFE is especially problematic when using GFP in monitoring transgenic plant cell suspension cultures, due to the aggregated nature of the cells and the high biomass concentration in these culture systems. Reported approaches for online compensation of IFE in monitoring culture NADH fluorescence or bioluminescence require online measurement of biomass density or culture turbidity/optical density, in addition to fluorescence/bioluminescence measurement. In this study, culture GFP fluorescence was used successfully to estimate GFP concentration and other important states in bioreactor culture of transgenic tobacco cells, while the influences of IFE and culture autofluorescence were rectified without the need for an additional biomass sensor. This was achieved by setting up a novel model-based state observer. First, we developed an improved model for a backscatter fluorescence probe that takes into account the influence of IFE and autofluorescence on reporting culture GFP concentration from online fluorescence. The state observer was then established using the extended Kalman filter (EKF), based on the fluorescence probe model, a dynamic state model of the plant cell bioreactor, and online GFP fluorescence measurement. Several versions of the observer were introduced to address practical requirements associated with monitoring GFP fluorescence of plant cell cultures. The proposed approach offers an effective means for online compensation of IFE to enable quantitative interpretation of the culture fluorescence signals for accurate reporting of GFP or GFP-fusion protein expression. (c) 2005 Wiley Periodicals, Inc.