Abstract: Inspired by the noise reduction capability of the serration at the owl wing trailing edge, this paper proposes two serration noise reduction structures applied to the sail trailing edge of underwater vehicle, namely, the triangular serration and the iron serration. This paper investigates the effect of serration on the flow and far-field noise characteristics around the sail by using LES and FW-H equations. Quadrupole noise is captured by constructing permeable integral surface combined with “Collapsing-Sphere” equations. The main coherent structures of the sound source are extracted based on the flow field reduced order method, and mapping relationship between the flow field and the sound source is identified. Thus, this reveals the physical mechanism by which trailing edge serrations affect flow noise. The results show that serrations have a double effect on the flow noise, which is the coupled effect of dipole and quadrupole noise. The triangular serration reduces the total noise by a maximum of 4.32 dB. The trailing edge serrations block the two sides of the flow from converging at the top and produce flow separation, reducing the turbulent fluctuations in the boundary layer. This destroys the vertical coherence and energy of the pressure pulsations, leading to the dissipation of the dipole noise source. The trailing edge serrations extend the continuous vortex structure in the sail wake stream, and the distortions in the spatial vortex structure increase the Lamb vector amplitude, resulting in a stronger quadrupole noise source energy.