fmod: See https://www.fmod.com/licensing#indie So basically yes free for small projects, put their logo somewhere, and you don't get source code or direct support from them. It does have a lot of features built in. If you wanted to distribute the engine to third parties, my understanding is you can't distribute FMOD binaries, they need to get their own licence and download themselves individually. You might ask sales@fmod.com..
You can use just some parts of SDL, for example you wouldn't need the image loading or text rendering libs. SDL audio is fairly basic though, basically just playing basic sounds/music “as is”.
I believe Creative still owns OpenAL. It hasn't really gone anywhere. OpenAL-soft provides a free LGPL software implementation of the API specification. I am not sure what the status is of even trying to get hardware support working even on Creative cards, but practically most audio is software these days.
On Windows XAudio2 is a nice option that full-fills a similar role of mixing multiple audio sources, dealing with different frequencies, 3D channels, DSP effects, etc. I found it really straight forward, at some point I intend to revisit OpenAL.
For OpenAL and XAudio2 loading various file formats etc. is mostly up to the user, they just provide powerful audio processing. Personally I found wav files pretty easy, as is Ogg Vorbis and I suspect the other Ogg audio libs. A little bit of plumbing code to move audio samples from one to the other.
Then obviously you have the actual OS level interfaces. On Windows these days there is WASAPI and on Linux I believe PulseAudio. These are however much more basic, any mixing/effects you need to do first. OpenAL, FMOD, SDL, XAudio2, etc. send their final outputs to these.