In this article, a video-based analysis for facial skin appearance with an automatic face tracking technique is proposed. The change of skin appearance under various illuminations observed by illuminating the face from various light source positions, and the changes in facial appearance are recorded (is a video stream during this measurement. The recorded video stream is then analyzed to obtain the bidirectional reflectance distribution function (BRDF) (it each position of the facial skin. Automatic face tracking is necessary to compensate the facial movement in order to obtain an accurate BRDF measurement at each position. Since conventional face tracking techniques cannot be used for a face illuminated from various light source positions, a new face tracking technique that is robust to the various face shadings has been developed. Experimental studies show the effectiveness of both the proposed video-based analysis technique and the automatic face tracking technique. (c) 2008 Wiley Periodicals, Inc. Cot Res Appl, 33, 477-484, 2008: Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/col.20440