architecture – Designing persistence in an ECS world subdivided into chunks

I’m designing a kind of simple open world with ECS. The whole world is too large to be loaded at once, so I load and unload chunks according to player’s position. Nothing fancy, pretty much standard nowadays.

In this world, I have entities that I need to update frequently. There are a couple of thousands of them. These entities share some components and a tag to retrieve them, but can refer to different archetypes. These entities are placed by hand, before runtime. No entity of this kind is created on runtime.

I need to update these entities every five minutes. I do not need them to update in a single frame, it can be scheduled over multiple frames (so performance is not exactly critical here but as always, the more performant the better).

  1. I need to update every entity, even if not loaded according to player’s position
  2. I need to save and load the state of these entities

Currently, the position of these entities is guaranteed to remain the same and it is guaranteed too that two entities can’t share a same position. However, it may change.

Solution A

I create a singleton “manager” that holds a container. Keys are the position of the entities. I iterate trough the container to update the values, so I don’t need to load entities.

I update the entities currently loaded with the new values retrieved from the manager.

Pros:

  • Fast
  • Easy to work with
  • No need to load entities (and meshes and so on)
  • Trivial to serialize and save
  • Do not require additional work on the world creation

Cons:

  • Not flexible as the position is now a key (could be a problem if I want the system to evolve)
  • May cause issues when parallelizing
  • Having positions as keys is probably a very bad idea
  • Defeats the principles of ECS as data is not “owned” by the entities, but is centralized

Solution A (alternative):

This time, with a script I execute before runtime, I set an ID to each entity in my world as part of a component. I use this ID to store the data in my container (which is now a simple array and not some sort of a dictionary).

Pros:

  • Very fast
  • Easy to work with
  • No need to load entities (and meshes and so on)
  • Trivial to serialize and save

Cons:

  • Does require additional work on the world creation
  • Still not very flexible (but better than positions)
  • May cause issues when parallelizing
  • Still defeats the principles of ECS as data is not “owned” by the entities, but is centralized

Solution B

This time, all the data remains in the entities. I have to separate my chunks into two parts: a part with the entities I need to update (let’s call that an update-chunk), a part with the rest of my entities (a regular-chunk).

I first have to create a save for my entities: I load every update-chunk, I update the entities, I serialize them then unload them.

Then, for each update: I deserialize my entities, I update the entities, I serialize them again.

When a chunk is loaded: I deserialize my entities and I load the regular-chunk only.

Pros:

  • I’m using ECS, yeah!
  • … Nothing much

Cons:

  • Ridiculously inconvenient and complicated
  • I need to serialize over and over again
  • Probably very, very bad performance
  • I save a lot of data I already “saved” in my base configuration (so many duplicates)
  • Difficult to set up as I have to create twice as much chunks
  • I have to load meshes/textures associated with the entities even if I do not need to render them (I can’t change this behavior in the framework I use, or I need to code another system and this is getting far too complex)
  • Probably more but that’s way enough to disqualify this solution

I’m pretty much convinced I’m doing something wrong and that I missed something really important.

Solution A may perfectly work (after all, I have no need to absolutely use ECS for everything) but it bothers me that I can’t figure out how to solve this problem with ECS.