DC magnetic susceptibility measurements on CsYbZnSe3 show a broad magnetic transition at approximate to 10 K and pronounced differences between zero-field-cooled and field-cooled data that lead to experimental effective magnetic moments of 4.26(5) BM and 4.39(4) BM, respectively. Specific heat measurements confirm that there is neither long-range ordering nor a phase transition between 1.8 and 380 K. First-principles electronic structure calculations with and without inclusion of spin-orbit coupling effects show that the spins of CsYbZnSe3 prefer to orient along [010] rather than along either [100] or [001] of this orthorhombic material and that the spin exchange between adjacent Yb3+ ions along [100] is substantially antiferromagnetic. The magnetic properties of CsYbZnSe3 are best described by an Ising uniform antiferromagnetic chain model.