In the field of civil and structural engineering, seismic design incorporating shape memory alloys has shown promise in substantially reducing post-earthquake residual strains. In contrast to extensively studied NiTi shape memory alloys whose widespread use as structural material is hindered by their high cost and challenging processing, the advanced Fe-based shape memory alloys possess significant potential. For polycrystalline Fe-based shape memory alloys, texture is critical to relieve intergranular constraints on superelasticity and modify the ductility of grain boundary during deformation. In this research, the texture evolution in polycrystalline FeMnAlNi alloy by deformation and recrystallization was systematically investigated using the electron backscatter diffraction method. The 94.1% cold-rolled sample was predominantly composed of the FCC phase (94.6% in volume fraction), exhibiting a deformation texture mainly consisting of Brass and Goss components, and a small amount of the BCC phase. After primary recrystallization annealing, newly formed and grown BCC grains were constrained by K-S orientation relationship and variant selection. {433}<337> and {100}<011> BCC grains almost entirely occupied the sample annealing at 1000 and 1100°C. Subsequently, when annealing temperature exceeds about 1120°C, abnormal grain growth of {211}<011> BCC grains began to occur concurrently with the dissolution of FCC phase, leading to a change of main texture to {211}<011>. Furthermore, the colonies structure and high mobility boundaries probably contributed to the extremely high velocity of abnormal normal grain growth (4.0 × 10−5 m/s).