I am thinking about adding some normal detail maps at high frequency for very small surface details. I'm having trouble thinking of how to handle the blending correctly without using quaternions. If you just add the normal map and the detail map together, the normal map will get flattened out, which we don't want:

vec3 n = (normal + detail) * 0.5;

I'm thinking add the X and Y components to the normal and re-normalize:

vec4 n = normalize(normal + vec3(detail.x, detail.y, 0));

This isn't exactly correct, but it will give an okay approximation without distorting the main normal map too much. Can anyone suggest a better formula that is inexpensive to calculate?

Thinking about what a detail normal-map is likely to represent--an offset from the base normal-map, rather than the actual normal at each point--it seems to me that your second approach should be more or less correct.

The only thing that I might suggest is to apply a scalar to the detail normal-map in order to control the degree to which it influences the final result.

Some comparisons: https://blog.selfshadow.com/publications/blending-in-detail/​