Hello, again!
I decided to run a bunch of experiments on SH computations, which is why my response took a few days. One of these experiments to try to brute-force the normalization factors for each order. I did this by brute-forcing an irradiance cube map, then fitting the SH normalization factors using stochastic gradient descent to find the optimal values.
I took the bases (?) up to order 4 from Wikipedia: https://en.wikipedia.org/wiki/Table_of_spherical_harmonics#Real_spherical_harmonics
After a few seconds, it converges to something like the following values:
A0: 3.1424568
A1: 2.095541
A2: 0.78717023
A3: 0.0
A4: -0.1300998
These values are within ~0.02 of the values listed in https://cseweb.ucsd.edu/~ravir/papers/envmap/envmap.pdf at the top right on page 2:
A0 = 3.141593
A1 = 2.094395
A2 = 0.785398
A3 = 0.0
A4 = −0.130900
More exactly, the correct normalization values are:
A0 = PI
A1 = 2/3*PI
A2 = 1/4*PI
A3 = 0
A4 = 1/24*PI
Note that all these values are off by a factor of 4, so the correct values are actually:
A0 = 4*PI
A1 = 8/3*PI
A2 = PI
A3 = 0
A4 = 1/6*PI
I can confirm that these normalization values look very good. Using 4*PI as the normalization factor for all orders of spherical harmonics looks very wrong.
Using order 2 spherical harmonics (9 coefficients) indeed seems to be by far the most value for your money. Going up to order 4 with 25 coefficients (of which only 18 have a non-zero normalization factor) gives an almost identical result while more than doubling the computation cost.