When an application remembers information, this information is called state¹. State is a key element of every application. In its simplest form, state is represented as a local variable that can be read and written. In this article, we discuss how the latest State Management approaches use state to control different aspects of a web application, such as the text inside a user interface², and what they might miss in the context of 3D and XR. We'll use the state of a simple game to illustrate the different state management approaches.

let health = 3
let healthText
function damage() {
    health--
}
function heal() {
    health++
}
function onFrame() {
    healthText = `${health} ❤️`
}

In this example, the health variable represents the state, which assigns the healthText value before rendering each frame using the onFrame function. Since new frames are rendered multiple times a second, which is typical for 3D and XR applications, the onFrame function consistently recomputes the healthText value based on the health state. However, reading the state, recomputing it, and reassigning dependent values at each frame causes a large computational overhead.

A performant alternative to recomputing values on each frame is to only recompute and reassign values when their dependent state has changed.

let health = 3
let healthText = `${health} ❤️`
function damage() {
    health--
    healthText = `${health} ❤️`
}
function heal() {
    health++
    healthText = `${health} ❤️`
}

This example performs better in most cases. However, the code is more complex and harder to maintain. For instance, when another developer introduces a respawn function, they must remember to manually invoke all recomputations depending on the changed state. Furthermore, if the state changes multiple times per frame, the illustrated solution is less performant.

The question of managing state is not unique to 3D experiences or games. Web developers also need a simple solution to consistently and efficiently manage state because they typically need to control complex user interfaces with a tight computational budget.

Local state

One state management approach used in many web frameworks, such as React, is local state. The idea is to scope state to a small part of an application and automatically recompute all values when any state inside or above its scope changes. React reduces unnecessary computations by allowing developers to explicitly notate the dependencies of a computation.

//local state in react
const [health, setHealth] = useState(3)
const healthText = useMemo(() => `${health} ❤️`, [health])
function damage() {
    setHealth(health => health - 1)
}
function heal() {
    setHealth(health => health + 1)
}

In applications with large interdependent states used across multiple parts of the applications, local state becomes very slow because it must be scoped to the whole application. This results in many unnecessary recomputations when, in reality, the state change only affects a small part of the application. In this case, even notating dependencies explicitly is only a partial solution because all dependencies still need to be compared each time the state changes.

Global state

Another approach for web applications with large states is global state. For instance, the library zustand composes everything in one big global state that can be read everywhere in the application.

//global state with zustand
const useZustandState = create((set) => ({
    health: 3,
    damage: () => set(state => ({ health: state.health - 1 }))
    heal: () => set(state => ({ health: state.health + 1 }))
}))

//somewhere in the application
const health = useZustandState(state => state.health)
const healthText = useMemo(() => `${health} ❤️`, [health])

Every time the state changes, every part of the application compares its current slice of the state against its previous slice.

const prevHealth = 3;
let currentHealth = 4;
const hasChanged = prevHealth != currentHealth

This approach is limited to primitive values, such as numbers, because data structures, such as arrays, have the same reference after their contents are modified (see Referential Equality). To solve this, performance is either lost by comparing the contents of the data structure or making the state immutable, which requires to shallow clone it every time the content is changed. The following example illustrates the cloning approach when using zustand.

//global state with zustand
const useZustandState = create((set) => ({
    healthMap: { user1: 3, peter: 2, gamerXXX: 4 },
    damage: (name) => set(({ healthMap }) => ({
        ...healthMap,
        [name]: healthMap[name]
    }))
}))
const healthMap = useZustandState(state => state.healthMap)

Applications with deeply nested state can use libraries like to simplify modifications to immutable data structures.

Fine grained reactivity

A recently popular approach mitigates the performance overhead of local and global states by using multiple small (global) states (e.g., atoms, signals, or runes) to achieve fine-grained reactivity. Each small state contains an immutable value. All parts of the application can consume the states and compose them together.

//multiple small states with signals
const health = signal(3)
const name = signal("user1")
const healthText = computed(() => `${name.value}'s ${health.value} ❤️`)
const damage = () => health.value = health.value - 1
const heal = () => health.value = health.value + 1

State management for 3D and XR

3D and XR applications typically use different data structures and have different requirements for performance. For instance, games often contain thousands of entries that must be updated on every frame. These games often accomplish this using an Entity Component System (ECS) to store, read, and update the state. The following example illustrates how to use the ECS miniplex to build a game with user entities.

const world = new World()

const user1 = world.add({
  health: { current: 3, max: 3 }
})
const peter = world.add({
  health: { current: 2, max: 3 }
})

const healthQuery = world.with("health")

function damage(entity) {
  entity.health.current -= 1
}

function heal(entity) {
  entity.health.current += 1
}

function onFrame() {
    //update healthText above all users
    for (const entity of healthQuery) {
        let healthText = `${entity.health.current} ❤️`
    }
}

As seen in the example, an ECS focuses on the performant traversal of entities with specific components, such as health, which is typically done on every frame. However, in this example, updating the healthText for all entities on every frame causes an unnecessary overhead when the health of one entity changes rarely. However, an ECS might be the most performant solution for recalculating the position of all entities on every frame. Therefore, it is reasonable to mix an ECS with a fine-grained reactive state manager by storing reactive states inside an entity and/or by storing the ECS itself inside the reactive state.

We propose to mix solutions like ECS with the innovations for state management on the web while extending them based on the following 4 specifics of state in 3D and XR applications.

1. Some state can change every frame

Many 3D and XR applications contain state that changes every frame, such as the current game time or the content of an ECS. Anything that depends on this state must be recomputed once per frame. In such a case, the overhead of checking whether a value has changed is unnecessary. The developer should, therefore, be able to denote that a state is live. When inserting an ECS into the reactive state and marking it as live, the calculations of the ECS are automatically executed on every frame.

In some cases, part of the state changes every frame and then does not update for a while. This behavior can be found in physics engines where colliders go to sleep once they stop moving. Therefore, a state manager should allow developers to specify whether a live state is asleep or awake to further reduce unnecessary recomputations.

2. Most state must only control other elements once per frame

Composing state requires combining the state changes of multiple dependent states. This can be achieved by recomputing the composed state as soon as one dependent changes. However, if another dependent state changes before the next frame is rendered, the first computation is unnecessary. As a solution, most recomputations can be collected and executed in one batch once before the frame is rendered.

Furthermore, computations are not distributed equally across all frames. Automatically scheduling the work to be executed across multiple frames allows to prevent dropped frames.

3. Some state can contain large data structures

If a state contains data structures, such as arrays, with a lot of entries, modifying this state typically requires computing the difference to determine what has changed. For instance, in a game with hundreds of players, adding a player to the state requires a shallow clone of the list of players, and determining the objects that must be added to the scene requires computing the difference between the previous and the current list of players. Instead of using a simple array inside the state, a solution is to make the array more intelligent, allowing it to track state changes, such as push or pop, alleviating the need to compute the difference.

4. Some applications render on demand

Some applications only render when things on the screen have changed to save battery power. A state manager can allow developers to support this without any additional work by demanding a re-render from the application every time the state changes. This also means that the state manager must demand new frames as often as possible if the state contains any awake live state.

Opportunity: Dev tools

A state manager made for 3D & XR has the opportunity to provide developers with more detailed information about potential performance bottlenecks. For instance, a state manager could inform developers about computations that are taking too long or warn developers when they write to a state on every frame that is not marked as live.

Conclusion

We believe that a state management solution that leverages awake/asleep live state, batching, scheduling, intelligent data structures and on demand frameloop will simplify how developers build performant 3D and XR applications that can be mixed with existing well-established solutions, such as an ECS.

References

¹ https://en.wikipedia.org/wiki/State_(computer_science)

² https://en.wikipedia.org/wiki/State_management