2nd run at shoreline, more to do.

https://scontent.fadl1-1.fna.fbcdn.net/v/t31.0-8/19095524_810386995808629_8161842409520699322_o.jpg?oh=328fec3ce81021511361cbb1faef9bd5&oe=59D7E72D

You have foam where waves are breaking, so that is what you want to simulate.

There exist other papers about this up to 10-15 years old, but I can't find the particular one that I have in mind (which was very good) right now...

This one however seems to describe the principle, too:
http://matthias-mueller-fischer.ch/publications/breakingWaves.pdf

The images in the paper are a bit coarse, you'd probably need 5-10 times as many particles, and have them textured in a way so there's a bit of a smooth falloff to make them look less artificial and to "multiply" them. Plus, texture the breaking wave body with a "foamy" texture on top. But in principle, that's it.