Inorganic membranes have been developed before 1945. The earlier application of inorganic membranes was primarily concentrate on military purpose. Carbon membrane is one type of porous inorganic membrane. Although the concept of carbon membrane for gas separation has been found in the early 1970, the interest to develop carbon membrane only increased, since Koresh and Soffer successfully prepared apparently crack-free molecular sieving hollow fiber carbon membranes. Nowadays, plenty of researchers have used different polymeric materials; including polyimides, to prepare carbon membranes by using pyrolysis. In general, carbon membranes can be divided into four major configurations: flat sheet, membrane supported on tube, capillary, and hollow fiber. Permeation properties of carbon membranes have been improved greatly in these 20 years. Carbon membranes offer advantages over polymeric membranes especially in terms of selectivity as well as thermal and chemical stability. More attention will be paid to carbon membranes in this century. This paper will review the development of carbon membranes in the last 30 years and give a clear future direction in research for carbon membrane.