Topological spin textures have attracted much attention both for fundamental physics and spintronics applications. Among them, antiskyrmions possess a unique spin configuration with Bloch-type and Neel-type domain walls owing to anisotropic Dzyaloshinskii-Moriya interaction in the non-centrosymmetric crystal structure. However, antiskyrmions have thus far only been observed in a few Heusler compounds with D-2d symmetry. Here we report a new material, Fe1.9Ni0.9Pd0.2P, in a different symmetry class (S-4), in which antiskyrmions exist over a wide temperature range that includes room temperature, and transform into skyrmions on changing magnetic field and lamella thickness. The periodicity of magnetic textures greatly depends on the crystal thickness, and domains with anisotropic sawtooth fractals were observed at the surface of thick crystals and attributed to the interplay between the dipolar interaction and the Dzyaloshinskii-Moriya interaction as governed by crystal symmetry. Our findings provide an arena in which to study antiskyrmions, and should stimulate further research on topological spin textures and their applications. Antiskyrmions had only been observed in a few Heusler compounds with D-2d symmetry. Here a new material, Fe1.9Ni0.9Pd0.2P, in a different symmetry class (S-4 symmetry) is reported to host antiskyrmions, as well as sawtooth surface textures and skyrmions.