So while looking at some other stuff, I figured I really should get this ECS stuff worked out, since it is not a pattern I have really used in any of my game stuff, or professional non-game software.

But there are many different approaches, and I am not clear on if there is any consensus on wrong and right ways to do things. Especially avoiding performance pitfalls if it was to be applied to large numbers of objects.

• A detail I keep coming across that I am definitely not liking is the use of maps (e.g. C# Dictionary or C++ std::unordered_map) in the Entity to store the components for that entity, sets to track either entities for a system or components of a type, etc.. Entities are added to and removed from systems when the components change. Then each update, for each system, the system iterates the entities in the set it is tracking. It must also then go through the component map to actually get those component instances, so that can be a lot of map lookups.
  
  Is this a good style, because I keep finding it but really not sure it is one I will like? Having the hash maps and sets definitely not something I like seeing. Often when profiling all types of application its the hash maps I end up finding on the hot paths, sometimes even with integer keys, and generally try to avoid them whenever possible.
  
  I have seen talk of not having an Entity object at all, instead have ordered arrays for each component so they can be iterated in step by systems to get a union (which could also allow some performance gains over OOP entity processing). Is this generally successful? And is there any examples?
  
  One thing I have been thinking of is that if each component instance has an EntityId and is stored ordered by it that such iteration is easy, but then deleting entities will be expensive because it would mean moving every component instance along after it along. Doing so would also invalidate any direct pointers to components I might want to have to avoid searching the component lists for Entity X repeatedly.

Which also brings the question of, what should not be part of the ECS in general?

• I am wondering what to do about other "objects" directly owned by entities. Should these "child objects" actually be entities in an ecs? I never had these in my entity hierarchy but there is some shared data and logic I handled using other non-entity classes? For example a ship may have several turrets, which do have their own positions, etc. I have generally made a Turret class or such to contain that, what their current target is, shooting, etc, just not one that ever inherited an Entity style class.
  
  Other things make use of this Turret class as well, such as tanks or buildings. An important note is that a Turret can not simply be a component, because an entity might have multiple turrets (ships being the most common). Other examples would be inventory items etc., which I only have in an "entity wrapper" when they are directly present in the world.
• Another object I've also kept separate is weapon projectiles. There is only a few major types and thus are pretty different, so I just coded them as their own things. These are much slower than real-life bullets, so can easily get large numbers, as well as a lot being created and destroyed. But in theory could use a common position, graphics, velocity, collision, etc. component as other things.
• Particle systems. Particles have even larger numbers, potentially with dozens existing for every projectile, so I am thinking there is no way those can be part of an ECS? The particle systems though again I am not sure, as they do have an origin position, take part in rendering, etc.
• Terrain tiles. Vast numbers, feels like this should still be its own thing, even in a compact format, it adds up, and can make good use of special render logic.

•  Stuff that is not world objects. Should these be part of the ECS? For example I have sound source objects to allow for the position to be updated by moving objects (non-moving and non-looping sound sources can just be discarded and the underlying sound instance will get cleaned up when the sound completes), so that could be a position component along with another component for the actual sound data? GUI elements also come to mind, but I feel that should almost certainly be left in its OOP hierarchy and tree reference structure.

I liked the example by Intel a lot.
https://software.intel.com/en-us/articles/get-started-with-the-unity-entity-component-system-ecs-c-sharp-job-system-and-burst-compiler

It's simpel enough to follow but shows well how all works together (at least for this case).

From the example I understood that it's main use is to separate often accessed data from its "surrounding waste". Giving you a way to help the CPU optimize your caches and data access.

So for now I try to bring stuff like projectiles, enemy position/state, healthbars into the ECS. Everything I have to access/update often but only process/modify limited data on.

But maybe (likely) I got this completely wrong.

I have started experimenting with ECS just now too. I want to address both Data Oriented Design and Entity-Component System. I really like the pattern with no Entity object, and entities being just unique numbers.

The following article describes both of these goals in detail: http://gameprogrammingpatterns.com/data-locality.html

It may or may not be a helpful resource, since it's a beginners introduction, but the very first page already hints at how to avoid this: SPECS

3 hours ago, SyncViews said:

I am definitely not liking is the use of maps (e.g. C# Dictionary or C++ std::unordered_map) in the Entity

It does not have a map in the entity. The entity is just an ID (+generation if the ID was already used but deleted and a new Entity with the same ID gets created).

Caveat: It's in Rust, but the basic idea should work in other languages,too.

How it works:
- Every component type defined ( independent from how often it is used ) has a vector containing all ID's of the entities that have a component of this type.

- you can `.join()` different components together (using a BitSet), and for all involved components the vector described above is checked, 

and if an ID shows up in every ID-vector, the corresponding entity is going to be manipulated by the system (means the Entity has to have ALL of the Components involved, but can have more).

EDIT: Fortgot the Link, thanks @SyncViews

30 minutes ago, Iltis said:

It may or may not be a helpful resource

Which article / resource are you referring to?

1 hour ago, GWDev said:

I liked the example by Intel a lot.

Will need to look at that some more, as it is not immediately clear how it is tracking the components to pass the IJobProcessComponentData

54 minutes ago, turanszkij said:

The following article describes both of these goals in detail: http://gameprogrammingpatterns.com/data-locality.html

From what I can tell just reading that now, that is using the "entity has a map of its components" approach. Only rather than storing an actual map, it explicitly coded in each component so that that O(1) add, fetch is a nice cheap direct access rather than a O(1) hash map. Having a "GameEntity" with every possible component seems like a bad idea.

The "for (int i = 0; i < numEntities; i++)" loops also wouldn't work if some of the entities did not have all the components in question... So needs some expanding, it seems to suggest having different entity types with their own pre-allocated component lists, for each supported combination? Not sure how that would scale.

54 minutes ago, SyncViews said:

From what I can tell just reading that now, that is using the "entity has a map of its components" approach. Only rather than storing an actual map, it explicitly coded in each component so that that O(1) add, fetch is a nice cheap direct access rather than a O(1) hash map. Having a "GameEntity" with every possible component seems like a bad idea.

The "for (int i = 0; i < numEntities; i++)" loops also wouldn't work if some of the entities did not have all the components in question... So needs some expanding, it seems to suggest having different entity types with their own pre-allocated component lists, for each supported combination? Not sure how that would scale.

 

No, this is describing a system where entity is just a number. Then we also have component managers/collections for every possible component type. We can add a component to componentmanager with an entityID, and that means that an entity received that component. The component manager itself can decide how it wants to implement looking up a component by an entity ID. You don't have a "GameEntity with every possible component", only if you registered the entity to every possible componentmanager. 

The "for (int i = 0; i < numEntities; i++) loops" are iterated against a specific componentmanager of your choosing, which will iterate through every component, regardless of what entity owns it.

Which bit are you looking at? This here is assuming that an entity at index i is valid for all the components that GameEntity contains, with the components at the same exact index.

Quote

The data is all there, one byte after the other. The game loop can then walk these directly:

while (!gameOver)
{
  // Process AI.
  for (int i = 0; i < numEntities; i++)
  {
    aiComponents[i].update();
  }

  // Update physics.
  for (int i = 0; i < numEntities; i++)
  {
    physicsComponents[i].update();
  }

  // Draw to screen.
  for (int i = 0; i < numEntities; i++)
  {
    renderComponents[i].render();
  }

  // Other game loop machinery for timing...
}

 

It also seems to not be covering how RenderComponent is getting the position (presumably from Physics), especially if the entity for physicsComponents[x] is not the same as renderComponents[x] for some entity x.

I recently also started to work with ECS, so I think I could add something to discussion. I'm using type maps for entity components. But doing lookups every update quickly becomes performance issue. To get rid of this problem, I allowed systems to pool components they need, so I'm observing when new entity is added or removed from system and then I copy it's components to the local pool. 

Quote

Other things make use of this Turret class as well, such as tanks or buildings. An important note is that a Turret can not simply be a component, because an entity might have multiple turrets (ships being the most common).

Child - parent could be problematic in ECS, but with your turret example, You could have for example a Turrets component which you attach to ship, tank or whatever entity. In Turrets component You have an array of Turret entities (or id's of turret entities). Then You can have set of system for example:

UpdateShipTurrets where you update entities with Ship and Turrets component,

Update TankTurrets etc.

Also you could have system UpdateTurrets for entities with Turret component, where you update things common for every turret like ammo or reloading. What do You think about this approach?

How exactly are you pooling/caching the components? The idea I had was to have a preallocated array, to be cache friendly, and keep the arrays sorted by EntityId, I might then iterate them for a system like:

auto pos = positions.begin();
auto proj = projectiles.begin();
while (pos != positions.end() && proj != projectiles.end())
{
  if (pos->id < proj->id) ++pos;
  else if(pos->id > proj->id) ++proj;
  else
  {
    // Projectile move logic...
    ++pos;
    ++proj;
  }
}

So should be a nice O(n) and scan through memory in a cache friendly way.

If entity ids always increment and all components are added upfront creation is just an append.

The issue here is how to delete stuff. Shifting everything forward to fill the gap would be expensive, and prevent anything else storing direct references leaving them with binary search (e.g. a homing missile getting the Position component for the target).

Additionally choosing the size in advance could be problematic.

IMHO, "ECS" is just a dumb rediscovery of the relational model, but with decades of computer science rigor discarded... https://en.m.wikipedia.org/wiki/Relational_model

Component types are tables. Components are rows. Entities are keys. Systems are stored procedures. 

Study the actual sciences RDBMS and object modelling techniques and you'll be an "ECS" expert.