I’ve got a hobby game that I’ve been using as a framework for learning graphics programming. It’s kind of a hybrid 3D modeler and sandbox simulation game, so there’s a fair amount of dynamically generated geometry.

I’m struggling to design my basic game loop and data structures. I am attracted to the ECS paradigm for its tendency to resist spaghettification. But I’m having trouble understanding how dynamically generated data fits into this pattern.

The simplest example that confuses me is a particle system. Let’s say I have an entity that can move around and also emits particles. The entity’s motion path generates a spline, which has to be recorded somewhere. It’s easy enough to imagine a ParticleEmitter component that records N previous object positions.

The first difficulty appears when I want to have the particles collide with the scene. It feels incorrect to model each particle as an entity (my camera sees a lot of the scene at once, so there could be thousands of particles spawned or killed on any individual frame), but I also feel like it’s incorrect for code cleanliness to teach the physics system about particles as a special case.

So it seems the answer is to have the particle system emit collision volumes without associating an entity to each collision volume. I guess I could generalize the Collision component to represent a collection of rigid bodies, and attach a single Collision component to the same entity as the ParticleEmitter. But the resulting collision geometry is dynamic, which means it can’t fit in the fixed-size Collision component.

Furthermore, the Physics system is going to update the position, velocity, and inertia of each particle separately, and probably needs a persistent BVH data structure to do so efficiently. Where does *that* data live? If the Collision component just points into the Physics system’s internal data structures, doesn’t that require the ParticleEmitter know at least enough about the Physics system to ask it to create collision volumes?

Kleriq said:
Furthermore, the Physics system is going to update the position, velocity, and inertia of each particle separately, and probably needs a persistent BVH data structure to do so efficiently.

If so, the efficient solution would be some setup work at the creation of the particle system: Calculate the bounding box of the spline, extruded by the maximum distance a particle can travel, then do a range query on the BVH of static scene using this box, and cache the traversal stack. This way all particles could start from that, saving a lot of costly traversal work (actually the number of particles, times particle system lifetime in frames, times traversal steps to build the cache).

That's responsibility of the physics system, so lower level than a ECS used to implement game logic, imo.

Edit: To clarify - the physics system should implement the particle system, which then ECS could represent as a single component. But i see no need to represent individual particles as game objects.

My concern is this logic leads to big blob systems. For example, the audio system needs to know about geometry to do acoustic modeling. Does that mean that the physics system should also handle audio?

Kleriq said:
Does that mean that the physics system should also handle audio?

I would say the physics system implements raytracing, which the audio system is using to figure out it's spatial model. (no idea how common it is to raytrace for audio, but just to make an example.)
It's also possible the audio system requires it's own representation of geometry, e.g. a density / SDF volume. Then ofc. physics triangle raytracing would not help, so no connection between those systems.
We could also make scene BVH, raytracing, or density volumes their own systems. But that's just a rhetorical question, because a well designed physics system will allow to use it's raytracing BVH stuff also without simulation anyway.

Kleriq said:
My concern is this logic leads to big blob systems.

What do you mean with ‘big blob?’

And if the concerns are justified, how would you implement the traversal stack optimization i've mentioned if the particle system is not part of the physics system?
The only option would be to expose low level inner guts of the physics systems spatial acceleration structure. But if you do this, you can no longer update the acceleration structure code without breaking the particle system.
So that's bad, and we're left with not doing the optimization at all, accepting a significant loss just to stick at some design paradigm, which you are not really convinced about in the first place.

Kleriq said:
For example, the audio system needs to know about geometry to do acoustic modeling. Does that mean that the physics system should also handle audio?

Since this is my particular area of expertise, I can offer some insight. Generally, we tend to have a separate copy of the meshes for acoustic ray tracing, rather than use some other system (e.g. physics) to do the ray intersections.

This is done for performance reasons (can use a very fast optimized BVH, no indirect function call overhead), and because the physics geometry may be incorrect for audio use (e.g. imagine invisible walls affecting the acoustics). The audio simulation may also be done asynchronously on a separate thread from physics/graphics, so it would be a bad idea to make thousands of async callbacks into the physics system from the audio thread. Better for audio to manage its own scene representation that mirrors the physics/graphics scene.

We may also want to use a much simplified mesh compared to the one used for graphics. Usually the acoustic mesh would be derived from some subset of the graphics geometry, then pre-processing and simplification applied. Using a separate mesh also decouples the acoustic material properties from the physics and graphics ones, giving more flexibility to the sound designer.

One way this can be implemented is to add an "AudioGeometry" component to any entity that should be part of the acoustic simulation. The geometry component might be initialized with the meshes for any sibling/child graphics mesh components. The audio mesh might be preprocessed and the result serialized somewhere so it can be loaded at runtime. At runtime, the audio geometry component informs the audio system of a new position of the geometry whenever it moves.

Better for audio to manage its own scene representation that mirrors the physics/graphics scene.

If we assume that the Physics component is capable of modifying the geometry (e.g. moving objects should affect acoustic modeling), how would you get this representation from the Physics component to the Audio component? It seems inefficient to serialize it from the Physics system’s internal representation back into components, but it also seems wrong for the Audio system to directly ask the Physics system to hand over its internal representation.

Kleriq said:

If we assume that the Physics component is capable of modifying the geometry (e.g. moving objects should affect acoustic modeling), how would you get this representation from the Physics component to the Audio component? It seems inefficient to serialize it from the Physics system’s internal representation back into components, but it also seems wrong for the Audio system to directly ask the Physics system to hand over its internal representation.

This can be done the same way as you would control graphics by the motion of physics objects, typically by copying the transform matrix from one system/component to another. I have never done anything more complex than that. Usually I assume that the scene is composed from instanced static meshes with dynamic motion, and this allows for modeling most things (e.g. doors, vehicles).

Aressera said:
This can be done the same way as you would control graphics by the motion of physics objects, typically by copying the transform matrix from one system/component to another.

I guess you also react to collisions, detected by the physics system? And if it's a sliding contact with friction, play a different sound than if it's just a short impact?
Not knowing much about audio, i'm pretty curious if there is some standard way to fake spatial audio, if you can talk.
For example, i guess if a source is not ‘visible’, you may reduce volume and add some reverb or low pass filter.
But how do you handle partial visibility? E.g. In one frame i can see the sound source through an open door of my room. In the next frame, the source moved just so much it's blocked from the wall. How do you know it is still good to hear, just a bit less?
I could imagine to use random paths similar to path tracing, but that's expensive, while probably still a hopeless trial to simulate real world acoustics?

One way this can be implemented is to add an "AudioGeometry" component to any entity that should be part of the acoustic simulation.

Usually I assume that the scene is composed from instanced static meshes with dynamic motion, and this allows for modeling most things (e.g. doors, vehicles).

So do you assume that the AudioGeometry component just holds a pointer to some sort of mesh resource?

I just finished watching a fantastic video from a AAA game developer who validated a lot of the unease I have about how ECS seems to encourage either storing or duplicating data in a way that’s not efficient for systems to operate on. In the second half of the (2+ hour!) video, he talks about how his personal toy engine organizes data in an ECS/GameObject hybrid.

In his system, components are stored directly on entities, and can have spatial relationships and hierarchies. This is great for things like attaching one mesh to another (e.g. an object held in a character’s hand), but it’s still unclear to me how this system deals with synchronizing multiple systems’ cached/optimized representations of the same data. But that problem has almost certainly existed since scene graphs were the new hotness.

JoeJ said:
I guess you also react to collisions, detected by the physics system? And if it's a sliding contact with friction, play a different sound than if it's just a short impact?

That's possible, but more of a sound design thing. There are more sophisticated methods based on actual simulation (see the work of Doug James @Cornell), but these tend to be relatively expensive for real-time use and hard to integrate into existing sound designer workflows. Realistic physics-based sound synthesis in games is probably at least a decade away.

JoeJ said:
Not knowing much about audio, i'm pretty curious if there is some standard way to fake spatial audio, if you can talk. For example, i guess if a source is not ‘visible’, you may reduce volume and add some reverb or low pass filter.

First there is a distinction between the term “spatial audio”, which specifically refers to simulating the directional characteristics of sound arriving at the listener's head in a free-field condition (no scene geometry), and “room acoustic simulation”, which considers the effect of the environment on the sound. For spatial audio, the standard way is to use a head-related transfer function (HRTF) as a filter for the audio. This works well (if the HRTF is a good match for you) and is reasonably efficient.

As for the problem of occlusion/diffraction that you mention, it's very difficult to do efficiently in a realistic way that works for all types of scene geometry. For an example of the recent state of the art, see our SIGGRAPH paper from last year. It's much more difficult than path tracing, because the rays can “bend” around objects. It needs to be spatially smooth and handle low-quality input geometry. Most geometric approaches do it by “edge diffraction”, where a sequence of diffracting edges is found between the source and listener, and then the sound pressure propagating over these edges is calculated using some diffraction model like the Uniform Theory of Diffraction (UTD).

There are other methods based on comparing the length of the direct path between source/listener (neglecting geometry), and the length of the shortest path around the obstructions. The longer the shortest path is in comparison to the direct one, the greater the low pass filtering. But then you are faced with the non-trivial task of computing the shortest path in a spatially smooth way. This path lies along edges of the scene geometry, so you are still stuck with doing edge diffraction.

For the reverb itself, that can be handled using a path tracing algorithm, similar to those used in graphics rendering. The main differences are that each ray keeps track of the propagation delay time, and that rays tend to bounce many more times (e.g. 200 bounces rather than just a few). Rather than an image, a spatial room impulse response is generated. This is a filter that gets convolved with the source audio. The main difficulty here is to make it have sufficient quality while running in real time. I developed a few techniques for speeding things up in my PhD thesis, which mostly involve caching/post-processing the results to mitigate Monte-Carlo variance.