If all voxels have the same geometry(cube) and AO doesn't depend on lighting, wouldn't it be possible to bake AO for a single voxel and reuse it for all? E.g. bake normal edge AO and 90 degree angle AO and just somehow calculate where to put it. Might be some more edge cases, but idea stays the same. This way we could get high quality AO without sacrificing performance.

The issue with this is that AO depends on a cube's surroundings. Now, I reckon you could pre-calculate the AO for all the possible surroundings patterns. Say, if your occlusion rays extend no more than 5 cubes away, you could pre-calculate the AO for a given cube using all possible 11x11x11 neighborhood patterns and attempt to build a look-up table. But the larger your AO neighborhood becomes, the more difficult this becomes.