DEM has been well established as a powerful numerical technique to study the fundamentals of granular materials. It has limited application in dealing with practical particulate systems which often involve billions of particles. Graphical Processing Units (GPU) offers a solution to this problem.This paper explores GPU-based DEM development, and its performance is assessed under various conditions. For a packing case using 300,000 spheres, it is demonstrated that the speed up ratio (single GPU to single CPU) varies from 40 to 75, depending on the parallel algorithms used. The GPU-based DEM is then developed for other systems involving arbitrary wall geometries, moving wall boundaries, and non-spherical particles. Particle flow patterns in the top charging system of blast furnace, screw conveyor and rotating drums are studied. For ellipsoids in the rotating drums, when aspect ratio increases or decreases from 1.0, the dynamic angle of repose increases. Multiple GPUs with message passing interface (MPI) are also used. It is demonstrated that the speed using 32 GPUs can be 18 times faster than a single GPU. It can handle large granular systems with more than 10 million particles. Thus, the GPU-based DEM developed in this work makes the simulation of real industrial processes possible. (C) 2016 Elsevier B.V. All rights reserved.