Building an ECS: Data Oriented Hierarchies
Should hierarchies be integrated with an Entity Component System, or should they be designed separately? It’s a question that has invited strong opinions ever since Flecs added hierarchies in 2018. Since then though, many other libraries followed:
- Unreal Mass added a relationships API.
- Bevy (the most popular Rust game engine) has a hierarchy implementation that is integrated with the ECS.
- Jecs (the most popular Roblox ECS) has native hierarchy support.
- BitECS (the most popular JS ECS) has native hierarchy support.
- Many others, like Ark, Fennecs, Friflo, Gaia ECS, StaticEcs, …
What can we learn from this?
The least controversial take is probably that having an ECS with support for hierarchies is useful. Many games need hierarchies, and having them as a stable, tested, out of the box feature is appealing to developers.
A more controversial take is that if hierarchies are so often used together with an ECS, then they should be part of the core ECS model. Aside from entities, components and systems, we should also be able to express (amongst others) parent-child relationships.
Many would argue that this goes beyond the responsibilities of an ECS, and that it’s better to stick to the basics. But who decided what those basics should be? If we trace back the genealogy of ECS, we invariably end up at either relational databases or first order logic. Both of these have one thing in common: relationships are a fundamental part of the data model.
At the end of the day though we’re here to make games. So let’s leave that historical baggage behind for a moment, and take a more objective look at whether integrating hierarchies with ECS makes sense. One way to do that is by looking at the usability and performance of an integration:
Usability
The usability of a hierarchy is important as it directly impacts how game content is authored. Authoring tools generally provide two kinds of hierarchies:
- Asset hierarchies
- Scene hierarchies
Asset hierarchies are used to design the different kinds of objects in a game. Here’s an example of a small asset hierarchy:
Asset hierarchies let us express inheritance relationships between different asset classes, often combined with features like property overriding. Examples of asset hierarchies are Unity prefabs and Godot inheritance.
Common features of asset hierarchies are:
- Create assets with default property values
- Create asset variations
- Override property values on asset instances or asset variations
- Add children to assets and asset variations
Scene hierarchies are used to express parent-child relationships between entities in a scene. Here’s an example of a small scene hierarchy:
Common features of scene hierarchies are:
- Instantiate assets
- Recursively despawn entities
- Reparent children
- Iterate the live hierarchy
- Lookup entities by name
For a hierarchy to score “great” on usability, it must support both asset and scene hierarchies. They should work well together too, as the nodes in an asset hierarchy are typically scene hierarchies.
Performance
The second axis we want to optimize is performance. This is especially important for an Entity Component System, as performance is one of the main reasons projects use an ECS. This brings us to data oriented design.
Data oriented design, or DOD, is the discipline of identifying how data flows through an application, and designing data structures that make optimal usage of the underlying hardware to enable those flows.
This can get very detailed, like optimizing datatypes to eliminate padding bytes or making sure that a struct fits inside a cache line. There are also higher level principles that, when followed consistently, generally lead to good outcomes like:
- Data that is accessed together should be stored together
- Write code that operates on collections, not single things
Pretty abstract stuff, so lets bring it down to earth with an example. If you already know DOD, you can skip this section.
This is fairly typical object oriented code. We design classes that represent “real world” objects which contain the properties that such an object might have. At first glance this sounds pretty good for a game engine, where often the goal is to simulate real-world objects.
… except this is not how the hardware prefers it. There are two problems here. The first problem is that each object has its own allocation (the new keyword in C++, malloc in C). Over time, this causes objects to become fragmented, meaning they are stored in many different memory locations.
Where this becomes an issue is when we want to move() many shapes at the same time. The hardware will have to work hard to fetch all the different RAM locations that have shapes. Fortunately, this problem has a simple solution: store the shapes inside an array instead:
Now all the shapes are stored together, and for reasons that I won’t go into here (but have done elsewhere), this performs much better. This approach is often referred to as arrays of structures (AoS).
We could call it here and get decent performance, but there is another issue lurking. The move() function needs Position +Velocity, while the draw() function needs Position + Color. Yet with this code, we will always load all properties in the CPU cache.
In our example we only have three properties so no big deal, but actual games can have dozens, even hundreds of properties. This would cause us to quickly exhaust the limited space in the CPU cache.
To fix this problem we have to let go of the object oriented model, and design our code around data instead. Here’s what that could look like:
This code creates an array per property, also referred to as structure of arrays (SoA). The move() and draw() functions now only load the data they really need, and data is still nicely packed together in arrays.
These are some of the many tricks that ECS libraries deploy to run games with thousands of entities. Most of this happens behind the scenes, hidden behind a high-level API that is similar enough to what game developers have come to expect from entity systems. This combination of usability + performance is what made ECS popular.
The real question is now: can we repeat this for hierarchies? Can we design an ECS integration that supports both asset- and scene hierarchies (already complex), and then apply data oriented design principles so that this integration is not only easy to use, but also performant enough for real, production-level games?
Can we design the integration so that it makes the ECS faster?
The next two parts of this article will not only show that the answer to this question is “yes”, but that deep integration unlocks new capabilities that allow for more creative expression!
To keep things honest, everything that this article describes is already implemented in Flecs, and is being dogfooded to make sure it actually performs as expected:
Part 1: Asset Hierarchies
To recap, asset hierarchies let us define which objects we can instantiate in a game. In Flecs this capability is provided by the prefabs feature. Let’s quickly go over what it looks like.
Here’s a simple prefab hierarchy in Flecs Script:
prefab Unit {
Health: {50}
}
prefab Warrior : Unit { // Prefab variant
Health: {100} // Component override
Range: {10}
// Child
weaponSocket {
Weapon: {}
}
}
my_warrior : Warrior { // Prefab instance
weaponSocket {
Weapon: {excalibur} // Override child component
}
}And here’s what that looks like when translated to C++:
entity Unit = world.prefab("Unit")
.set(Health{50});
// Prefab variant
entity Warrior = world.prefab("Warrior").is_a(Unit)
.set(Health{100}) // Component override
.set(Range{10});
// Child
entity weaponSocket = world.prefab("weaponSocket").child_of(Warrior)
.set(Weapon{});
// Prefab instance
entity my_warrior = world.entity().is_a(Warrior);
// Override child component
entity weapon = my_warrior.lookup("weaponSocket");
weapon.set(Weapon{ world.lookup("excalibur") });One thing you’ll notice in these examples is the repeated usage of inheritance (“is_a”) for creating asset variations and asset instantiation. How does this work?
In short, if entity A has an inheritance (“IsA”) relationship to entity B, we can access the components of B on A:
entity base = world.entity().set(Health{100});
entity instance = world.entity().is_a(base);
cout << instance.get<Health>(); // 100You may recognize this from prototype inheritance in Javascript:
const base = { health: 100 };
const instance = Object.create(base);
console.log(instance.health); // 100Just like with prototype inheritance, we can override components:
entity base = world.entity().set(Health{100});
entity instance = world.entity().is_a(base);
instance.set(Health{200}); // override
cout << instance.get<Health>(); // 200This gives us the bones of an asset hierarchy implementation. We just need to make sure that the asset entities aren’t picked up by our game logic. In Flecs this is solved by simply adding a Prefab tag to assets, lets queries and systems know to ignore these entities by default.
Because prefabs are part of the runtime state of a game, we can do things that are typically not possible in other implementations:
- Create and list asset at runtime (useful for editors / creative mode)
- Change the asset of an instance without changing the instance identity
- Associate an instance with multiple assets
- Change a shared asset property for all instances in constant time (see “component sharing”)
That’s good bang for our buck! But does it perform?
Asset Instancing
Game engines often need to group instances of different assets together. The most straightforward example of this is a renderer, as drawing instances of the same mesh in a single draw call (“instancing”) can greatly reduce the number of draw calls.
Engines sometimes achieve this with an “extract” step, which iterates the content in a scene, and populates buckets for the different assets that then get sent to the GPU:
In a naive approach this logic would look something like this:
auto q = world.query<Asset, WorldTransform>();
q.each([&](Asset& a, WorldTransform& wt) {
RenderBuffer& b = renderer.getBuffer(a.assetID);
b.append({wt});
});Simple, but this code needs a lookup per entity to find the target buffer. For scenes with a lot of entities this is wasteful.
We could solve this by adding a data structure that stores all entities per asset, but this just moves cost to another performance critical part of the engine (spawning), adds complexity, and increases memory utilization.
What if instead we use the existing ECS infrastructure to group entities by asset? Archetypes could help out here, as they already group similarly shaped entities together. We can organize our storage so that all entities in an archetype are of the same asset:
Not only does this get rid of the per-entity entity lookup: if we do GPU-side culling and cleverly model our data, we can memcpy the scene data directly into our renderer buffers! How could we write this code?
First we create a query that iterates all archetypes. Then for each archetype we lookup the asset, and then copy all entities in that archetype to the target buffer:
auto q = world.query<WorldTransform>();
q.run([](flecs::iter& it) {
WorldTransform& wt = it.field<WorldTransform>(0);
// Fetch assetID from archetype
flecs::table table = it.table();
flecs::entity assetID = table.target(flecs::IsA);
RenderBuffer& b = renderer.getBuffer(assetID);
// Iterate entities in archetype
for (auto i : it) {
b.append({wt[i]});
}
});This is a bit more setup than in our previous example, but we do now have a tight inner loop where we can simply append entities to a buffer.
We can optimize this code a little further with grouped queries. Grouped queries in Flecs make it possible to group matched archetypes by some value. In this case, we can use the asset ID as our group, which causes the query cache to look something like this:
With grouped queries we can rewrite the code to:
auto q = world.query_builder<WorldTransform>()
.group_by(flecs::IsA) // The IsA relationship points to the asset ID
.build();
// Iterate all groups (assets) matched by the query
for (auto g : q.groups()) {
RenderBuffer& b = renderer.getBuffer(g.first);
// Iterate all archetypes/entities for the current asset
q.set_group(g.first).each([&](WorldTransform& wt) {
b.append(wt);
});
}This works because IsA is a fragmenting relationship: entities with a different IsA relationship end up in a different archetype.
While in theory this increases the number of archetypes and therefore can increase overhead, in practice these costs are negligible as entities are very likely already fragmented on asset type. Different assets typically have different components, which means different archetypes.
So rather than increasing fragmentation, we merely guaranteed that entities of different asset kinds are in different archetypes. We then used that guarantee to optimize our downstream rendering code.
Component Sharing
Let’s now take a look what instantiated assets actually look like in memory:
For most games and assets, a lot of data does not change, and is duplicated for each instance. This is not great for a number of reasons:
- It wastes RAM
- We need more roundtrips between RAM and CPU caches
- Useful data is evicted faster from CPU caches
- Copying redundant data slows down asset spawning
Deduping this data is not always straightforward. Some instances may want to override these values. For example, an injured warrior may have a material with different textures, or a warrior with a temporary buff may override the Attack component. A mechanism that deduplicates data still needs to allow for per-instance overrides.
In Flecs this is addressed with shared components. Instances can inherit components from their assets. To enable this, we mark a component as inheritable:
world.component<Attack>().add(OnInstantiate, Inherit);If we do that for the static components in the above example, the storage changes to this:
Instances that override a component are stored in a different archetype:
When an instance “owns” a component that the asset also has, it effectively masks the component from the asset.
Flecs transparently resolves the correct component, whether it is owned or inherited. For example, the following system matches entities that both own and inherit the Attack component:
world.system()
.each([](Position& p, const Attack& a) {
// ...
});Finding the right inherited component requires iterating an asset hierarchy of possibly multiple levels deep. In a naive implementation this could easily become a bottleneck. However, we can observe that all entities in an archetype will inherit the same component. Therefore, the first optimization is to only do this search once per archetype.
Additionally, since queries cache archetypes, we can extend this cache to also store the result of this search. As a result the lookup cost of inherited components is therefore fully amortized, and practically free!
Another advantage is that when we now spawn instances, there is much less data to copy from the asset to the spawned entities, which is one of the biggest factors affecting spawn performance.
With the guarantee that instances of different assets are in different archetypes, we offload the asset bucketing usually done by a renderer to the ECS at minimal cost. With component sharing we can optimize spawn performance and significantly reduce CPU cache utilization.
Not only did we add new features to the ECS that now let us natively express asset hierarchies. In doing so, we made the ECS faster!
2. Scene Hierarchies
If you’ve used any popular game engine, you’re almost certainly familiar with scene hierarchies. They’re easy to understand, which can lead to the impression they’re also easy to build. Why not just do this and call it a day:
class Entity {
vector<Entity> children;
};But then you think of everything the scene hierarchy needs to do:
- Spatial relationships
- UI
- Grouping and organization
- Lifetime management
- Visibility management
- Change detection
- Entity disabling
- Event propagation
- Scene authoring
And it doesn’t stop there.
Some hierarchies have parents with thousands of children. Some hierarchies are 40 levels deep. Some hierarchies consist of only static objects. Others are fully dynamic with hundreds of joints that are updated every frame. For some hierarchies child order needs to be preserved. The speed at which entities can be spawned, despawned and reparented directly impacts how dynamic the content of a game can be.
Solving for all of these constraints is not an easy task. It is especially difficult in an ECS, since if we cannot do this efficiently, any performance gains from using an ECS can be easily nullified.
Can we do the same thing as we did for asset hierarchies? Can we integrate scene hierarchies with the ECS in a way that not only keeps performance benefits intact, but makes the ECS faster?
Let’s start with a simple premise: what if we, just like we did for asset hierarchies, introduce a fragmenting ChildOf relationship?
Fragmenting Scene Hierarchies
To quickly recap, if a relationship is “fragmenting”, it means that entities for different relationship pairs end up in different archetypes. In this case it means that entities for different parents end up in different archetypes.
At first glance this seems like a really bad idea. We could very easily end up with thousands of archetypes, and as a result our component memory would be scattered all across the heap in little bits and pieces. It worked well for asset hierarchies, but maybe not a great fit for scene hierarchies.
Or is it?
A fragmenting relationship essentially behaves the same way as a plain component. This means that everything we can do with components, we can also do with a fragmenting relationship, and for the same cost.
We can use the plain new/delete operations of an ECS to create and delete children. Reparenting is a single archetype move, where we encode on the archetype graph that adding a ChildOf pair for one parent removes the existing ChildOf pair:
Where fragmenting relationships shine is in how easy it is to query them. We can find all children of a parent for the same cost as querying for any component. We can also query for components of those children essentially for free:
// Just as cheap as a query for Position, Velocity
world.query_builder<Position>()
.with(ChildOf, parent)
.each([](entity e, Position& p) {
// ...
});Transform systems often need to iterate entities from top to bottom. We can use the same query grouping trick from before, but instead of grouping entities by asset type, we now group them by hierarchy depth. We can then make sure that the query stores the groups in sorted order, which essentially means we get breadth-first iteration for free:
This means we don’t have to sort entities, or even archetypes. We only need to sort groups whenever a new “depth” is introduced. This is a sort on a very small list, that happens very infrequently.
Additionally, since all children of an archetype have the same parent, we only need to fetch the parent component once. This means that the core transform loop just iterates contiguous arrays and can remain branch free. Here’s what that could look like:
auto q = world.query_builder<Transform, const Transform>()
.term_at(1).cascade()
.build();
q.run([](flecs::iter& it) {
auto t = it.field<Transform>();
auto pt = it.field<const Transform>();
for (auto i : it) {
// Look mom, no branches!
t[i].transform(pt);
}
});or the easier-to-write each variant:
auto q = world.query_builder<Transform, const Transform>()
.term_at(1).cascade()
.build();
q.each([](Transform& t, const Transform& pt) {
t.transform(pt);
});No need for a hierarchy walk. No need for sorting. No ad-hoc fetching of components. This code will outperform most transform implementations. Looks like we did it again. By integrating hierarchies with the ECS, we made our code faster!
Except, we did not really.
Consider what happens if a scene hierarchy looks like, well, what scene hierarchies usually look like:
Say this scene instantiates 1000 tank entities, where each is its own small hierarchy with a chassis, turret and tires. That means 1000 chassis children, 1000 turret children and 1000 tire children. With fragmenting hierarchies, where each parent gets its own table, that’s 3000 archetypes.
3001 if we’re being exact. We have one table that stores the 1000 top-level tank entities. This is where fragmenting hierarchies shine: hundreds or more entities for the same parent.
For the other entities though, having to churn through 3000 archetypes on most ECS systems is far from ideal. At that point cache misses start to hurt badly. It also doesn’t do spawn performance any favors, as we’ll have to create several archetypes with (ChildOf, tank) for each tank.
Back to the drawing board.
Non-Fragmenting Hierarchies
There is another straightforward option for implementing hierarchies in an ECS: a Children component, often in combination with a Parent component:
struct Parent {
entity parent;
};
struct Children {
std::vector<entity> children;
}Vectors-in-components are somewhat maligned in the ECS community, but if your ECS supports it they are totally fine. Especially since we trade in a little bit of memory fragmentation (the vectors) for fragmenting the entire component storage.
A hierarchy based on a Children component has another advantage, and that is that children remain ordered. This is especially important for UI entities, where you need to be able to control the display order of widgets.
If we now were to write a transform system for this new storage however, we get something that is considerably less efficient than the tight, branchless loop we had before:
void transformChildren(entity parent, const Transform& pt) {
const Children& s = parent.get<Children>();
for (auto e : s.children) {
Transform& t = e.get_mut<Transform>();
// fetching additional child components further increases cost per child
t.transform(pt);
transformChildren(e, t);
}
}Another downside is that removing a single child from a parent is an O(n) operation, where N=the number of children for a parent. The reason for this is that we need to find where in the Children vector the child is stored in order to remove it. For a parent that has 10 children this is fine, but not great if the parent has 1000 children.
A fragmenting hierarchy implementation does not have this problem, because an archetype ECS stores at which row of an archetype an entity is located. Therefore that row can just be removed directly, and we don’t have to find it first (for more details on the delete operation, see this article).
There is another solution that was proposed by the author of EnTT, where each child entity gets a component that links it to its parent and siblings:
struct Relationship {
flecs::entity first;
flecs::entity prev;
flecs::entity next;
flecs::entity parent;
};It is an elegant solution as it avoids dynamic allocations in components, and solves the O(n) child removal problem. Compared to the other approaches it probably has the least number of sharp edges, as performance stays consistent across essentially all hierarchy operations.
The downside is that this is a doubly linked list, with the performance tradeoffs that you’d expect from one. A single reparent operation writes up to 7 components, and iterating all children for a parent is- while constant time- several orders of magnitude more expensive than iterating a simple children vector.
This, in a nutshell, is the problem with scene hierarchies. There are plenty of locally optimal solutions, that when applied elsewhere become inefficient or even prohibitively expensive.
Hybrid Hierarchies
At this point we could just pick an approach based on the patterns we expect to be the most common, and let users deal with the edge cases where the implementation falls short. This is essentially what the big engines do.
As you can guess though, from a UX perspective this is not great:
- Dealing with multiple hierarchy APIs is inconvenient, and correct usage is difficult to enforce at scale
- Common tooling (like engine editors, the explorer) won’t be able to introspect home-grown hierarchies
- It’ll be more difficult to find and share online resources
- Maintaining, testing and refactoring an in-house hierarchy is expensive, especially once a significant amount of content is authored
Therefore, if we want a hierarchy that is both usable and fast, we’re left with just one option: we must go hybrid.
At least, that is the conclusion I arrived at after years of experimentation, and is what I ended up implementing for Flecs. Is it the most optimal solution? More on that at the end. First, an overview of how it works.
Here’s the traffic scene from earlier:
It has 40.000 moving cars that are all parented to a “cars” parent. Each car is a parent entity with 5 mesh entities: one for the chassis and four for the wheels. We’re intentionally not combining meshes or using LODs, as we want to stress test the hierarchy implementation.
Can any of the discussed approaches “solve” this scene? Not really:
- With fragmenting hierarchies we get at least 40.000 tables. Not good.
- With a
Childrencomponent, removing a single car is an O(n) operation where N=40.000. Also not good. - With the doubly-linked list approach, our transform system has to do 40.000 dependent loads at best. Won’t achieve 60 FPS unless you have a monster CPU.
That’s why Flecs has not one, but two hierarchy implementations:
- Fragmenting hierarchies
- Non-fragmenting hierarchies (similar to the
Parent+Childrenapproach)
If we take the traffic scene and implement it just with fragmenting hierarchies, we get crazy archetype zoo:
Queries matching chassis and wheel entities must iterate over each tiny archetype, where each archetype means cache misses. Contrast that with a setup where we switch these over to non-fragmenting hierarchies, and it’s easy to see why this is a lot faster:
How much faster? The traffic scene has in total 240.000 moving entities (6 for each car) and 750.000 static entities. When we run this, we get:
- 10 FPS, 1.5 GB if we just use fragmenting hierarchies
- 60 FPS, 500 MB for non-fragmenting hierarchies
That’s a huge improvement.
What about more complex scenes like the following example:
This entire scene is composed out of procedural assets that spawn just primitive shapes. Each unique color is its own shape, and there are in total 230.000 shapes organized in hierarchies that go up to 10 levels deep:
City -> City Block -> Building -> Floor -> Side -> Door -> Frame -> RectangleWhen we run this, we get:
- 23 FPS, 580 MB with just fragmenting hierarchies
- 90 FPS, 63 MB with non-fragmenting hierarchies
This enables an entirely new category of content creation that up to this point was not feasible.
Flecs augments the hierarchy implementation with two additional optimizations:
- Tables that store non-fragmenting hierarchies are annotated with a
(ParentDepth, N)value. This lets us iterate parents before children for free, in much the same way as thecascadefeature discussed earlier. - Prefabs with non-fragmenting hierarchies create a “tree spawner”, which is a small utility that speeds up spawning trees by caching things like the tables children will be created in.
This leaves little to be desired in terms of performance. But what about usability? Here’s what the above example could look like in Flecs code:
// Basic prefab setup (leaving out actual components for brevity)
entity car = world.prefab("Car");
world.prefab(flecs::Parent{car}, "Chassis");
world.prefab(flecs::Parent{car}, "Wheel_FL");
world.prefab(flecs::Parent{car}, "Wheel_FR");
world.prefab(flecs::Parent{car}, "Wheel_BL");
world.prefab(flecs::Parent{car}, "Wheel_BR");
// Create parent entity for all cars
entity cars_parent = world.entity("cars");
// Instantiate 40000 cars
for (int i = 0; i < 40000; i ++) {
world.entity(cars_parent, nullptr /* no name */).is_a(car);
}Click here for the same example in Flecs script.
The main API difference between hierarchies is how entities are created:
// Fragmenting ("ChildOf") hierarchy
world.entity(parent, "some_name");
// Non-fragmenting ("Parent") hierarchy
world.entity(flecs::Parent{parent}, "some_name");Once created, the rest of the API behaves the same, regardless of hierarchy implementation. Here are some examples:
// Iterates children for both hierarchies
parent.children([](entity child) {
// ...
});
// Returns parent regardless of hierarchy
auto parent = entity.parent();
// Matches entities for both hierarchies
world.query<Transform, Mesh>()
.with(flecs::ChildOf, parent)
.each([](Transform& t, Mesh& m) {
// ...
});In the vast majority of applications, the hierarchy kind will be specified on the prefab and prefab children. This means that for the most part, application code can remain hierarchy agnostic, and switching between hierarchies is as easy as changing a few lines of code. This is as close to ideal usability as we can get with a hybrid implementation.
To get the API to be hierarchy agnostic while not degrading performance for existing use cases is by far the most complex part of a hybrid implementation. It is largely why this took a year to implement in Flecs, and is accompanied by almost 1000 new test cases.
Hierarchies, solved?
So where does this leave us? Are hierarchies solved now, and should everyone implement this approach?
Well, no. Not if you want to follow data oriented principles.
Flecs implemented the non-fragmenting hierarchy storage as an answer to the limitations of the existing fragmenting storage. Each engine should do its own homework to find out where the gaps are. If nothing else, this article serves as a (very wordy) recommendation to carefully analyze usage patterns and performance before moving to an implementation.
There are also plenty of hierarchy-adjacent topics that this article does not address but can have a big impact on performance, such as:
- Reactivity (as seen in React, Vue, Svelte)
- Change detection
- Component inheritance
- Animation
- Entity disabling
- Event propagation
- Graph traversal
That will have to wait for another time. For now, congratulations, you’ve made it to the end of the (much longer than intended) article!
If you enjoyed the article, and want to see more like this in the future, consider giving Flecs a star!
(no 🤖 was used to write this article)
