Optimize Colyseus schema buffer usage by detecting and extracting shared properties from room objects, reducing data transmission size without losing functionality.

Scene data setup

The scene data filter is active by default in every room, a game maker does not need to enable anything. The administration panel is only used to decide how the rooms are filled and to turn the filter off while debugging.

1. Log in to the administration panel at /reldens-admin (the path can be changed with the RELDENS_ADMIN_ROUTE_PATH environment variable).
2. Open Rooms > Rooms (/reldens-admin/rooms). Every room in this list sends its scene data filtered when it is created, there is no per room option.
3. The filter saves the most data in the rooms with many objects that share the same sprite, like the enemies created by a respawn area (Respawn section) or several doors using the same asset key in their Client Params (Game Objects > Objects). Their shared values are sent once instead of once per object.
4. To send the complete data while debugging, open Settings > Config (/reldens-admin/config), click Create New and fill Scope server, Path rooms/data/sendAll, Value 1 and Type boolean, then click Save.
5. Restart the server: the config is loaded when the server starts. Set the value back to 0 (or delete the row) and restart again to filter the data in production.

Configuration reference

• server / rooms/data/sendAll (config, boolean, not installed) - when missing or 0 the scene data is filtered, 1 sends all the data unfiltered (debugging only).
• server/customClasses/sceneDataProcessor - a custom processor set by a developer in the server plugin, see Custom processor below.

Code integration (advanced)

Overview

The SceneDataFilter system prevents Colyseus buffer overflow by analyzing room data and extracting identical properties across multiple objects into a shared defaults structure. In the illustrative estimate of a room with 400+ objects (see Performance Impact below) it reduces the buffer usage from ~176 KB to under 64 KB.

Key components:

• Server: SceneDataFilter (lib/rooms/server/scene-data-filter.js) - detects shared properties, creates optimized data structure.
• Client: AnimationsDefaultsMerger (lib/game/client/animations-defaults-merger.js) - merges the preloadAssetsDefaults back into the preload assets and the animationsDefaults back into the objects.

Critical design principle: the filter NEVER adds properties to objects. It ONLY extracts existing identical properties to a separate defaults structure.

Server-side: SceneDataFilter

Architecture

Methods:

• filterRoomData() - main entry (called by State.mapRoomData). With sendAll: true it returns Object.assign({}, roomData) (unfiltered copy).
• buildCompleteData() - returns an unfiltered copy, not called by anything (the sendAll path of filterRoomData builds its own copy).
• buildFilteredData() - returns optimized data (sendAll: false, no custom processor).
• optimizeData() - generic optimization method.
• detectIdenticalProperties() - finds shared properties across objects.
• valuesAreDifferent() - compares values for optimization.

How it works

1. No hardcoded fields: the filter dynamically detects which fields are identical across objects.
2. Grouping: objects are grouped by a shared field for comparison (the value is resolved by GroupValueResolver, lib/game/group-value-resolver.js, shared by the server filter and the client merger):
   • preloadAssets: groups by asset_type (filters asset_type === 'spritesheet').
   • objectsAnimationsData: groups by the asset_key field, falls back to the key field if asset_key is not present.
3. Detection: for each group with 2+ objects, detects properties with identical values across ALL objects.
4. Extraction: identical properties are extracted to a defaults object, keyed by the grouping field value.
5. Grouping field preservation: the grouping field (e.g. asset_key) and the key field are removed from the detected identical properties and kept in each object so the client can look up the defaults.

Optimization logic

// Example: 200 objects with asset_key: 'enemy_forest_1'
{
  'enemy_1': {asset_key: 'enemy_forest_1', type: 'npc', enabled: true, x: 100, y: 200},
  'enemy_2': {asset_key: 'enemy_forest_1', type: 'npc', enabled: true, x: 150, y: 250},
  ...
}

// Filter detects: type, enabled are identical across all 200 objects
// Result:
{
  objectsAnimationsData: {
    'enemy_1': {asset_key: 'enemy_forest_1', x: 100, y: 200},
    'enemy_2': {asset_key: 'enemy_forest_1', x: 150, y: 250},
    ...
  },
  animationsDefaults: {
    'enemy_forest_1': {type: 'npc', enabled: true, ...}
  }
}

Key points:

• asset_key stays in each object (needed for the client to look up the defaults).
• Only properties with IDENTICAL values across ALL objects in the group are extracted.
• Properties with different values (x, y, content, options, id) stay in each object.
• Single-object groups are NOT optimized (no shared properties to extract).

When optimization happens and when it does not

Town room (reldens-new-age-town, 7 doors and 4 NPCs):

• The 7 doors share the door_house_3 asset key (their client_params.asset_key), so they are one group: the properties with the same value in every door go to animationsDefaults['door_house_3'], the different ones stay in each door (key, position and positionFix, because one door uses another offset).
• Each NPC has its own key and no asset key, so every NPC is a single-object group: no optimization, its data stays as it is with the key field as asset reference.

Forest room (illustrative example with 400 NPCs):

• The numbers are illustrative: the sample data reldens-forest-level-1 room has 12 Tree and 18 Tree Punch enemies and 10 mining rocks, and the reldens-bots-forest room has 100 Tree and 200 Tree Punch enemies.
• 200 enemies of type A, 200 enemies of type B.
• Each group has identical shared properties.
• Result: animationsDefaults: {'enemy_forest_1': {...}, 'enemy_forest_2': {...}}.
• The asset_key field comes from the object data built by the server, the filter only groups by it and never adds properties.
• Optimized objects contain only unique properties (x, y) plus the asset_key reference.

preloadAssets filtering

Process:

1. Filters only asset_type === 'spritesheet' (matches the client loader).
2. Groups the remaining assets by asset_type.
3. Detects identical properties across assets with the same type.
4. Extracts them to preloadAssetsDefaults[asset_type].

Typically kept fields (detected dynamically, NOT hardcoded):

• asset_type - grouping field (stays in each asset).
• asset_key - usually unique per asset.
• asset_file - usually unique per asset.
• extra_params - often identical for the same asset_type.

Typically moved to the defaults (if identical across assets): database metadata fields if they happen to be identical.

Result: minimal optimization for preloadAssets, since most fields are unique per asset.

objectsAnimationsData optimization

Process:

1. Groups objects by the asset_key field (or the key field if there is no asset_key).
2. For groups with 2+ objects: detects identical properties.
3. Removes the grouping field from the identical properties (keeps it in each object).
4. Extracts the identical properties to animationsDefaults[grouping_value].
5. Objects retain only unique properties plus the grouping field reference.

Grouping field priority:

1. Use asset_key if present (already set by the server for optimized objects).
2. Fall back to the key field if there is no asset_key (non-optimized objects).

Critical: the grouping field (asset_key or key) is NEVER extracted to the defaults. It must stay in each object so the client can look up the correct defaults entry.

Client-side: AnimationsDefaultsMerger

Merges the extracted defaults back into the preload assets and the objects after receiving the optimized data from the server.

When it runs

RoomEvents.checkAndCreateScene() (lib/game/client/room-events.js) runs it over the parsed scene data:

this.roomData = AnimationsDefaultsMerger.mergeDefaults(sc.toJson(this.room.state.sceneData));

mergeDefaults runs mergeGroupDefaults twice: first for preloadAssets with preloadAssetsDefaults grouped by asset_type, then for objectsAnimationsData with animationsDefaults grouped by asset_key. Each pass leaves the room data unchanged when its data or its defaults are not present. The server adds animationsDefaults: {} and preloadAssetsDefaults: {} (even if empty) when the filter is active.

Preload assets: when every spritesheet of the room shares the same extra_params (for example two enemies with the same frame size and no other objects), the filter moves extra_params to preloadAssetsDefaults.spritesheet. The merger restores it before ScenePreloader.preloadValidAssets() reads asset.extra_params, otherwise the spritesheets are never loaded and the objects have no sprite.

Merge logic

static mergeGroupDefaults(roomData, dataKey, defaultsKey, groupingField)
{
    if(!sc.hasOwn(roomData, defaultsKey)){
        return;
    }
    if(!sc.hasOwn(roomData, dataKey)){
        return;
    }
    let groupDefaults = roomData[defaultsKey];
    let groupData = roomData[dataKey];
    let itemKeys = Object.keys(groupData);
    for(let key of itemKeys){
        let itemData = groupData[key];
        let groupValue = GroupValueResolver.resolve(itemData, groupingField);
        if('' === groupValue || !sc.hasOwn(groupDefaults, groupValue)){
            continue;
        }
        groupData[key] = Object.assign({}, groupDefaults[groupValue], itemData);
    }
    delete roomData[defaultsKey];
}

Key behaviour

Optimized objects (their group value has a defaults entry):

1. Group value resolved with GroupValueResolver.resolve(objectData, 'asset_key'): the asset_key value, or the key value when there is no asset_key.
2. Merged: Object.assign({}, defaults, objectData) - object properties override defaults.
3. The result has all the properties needed for rendering.

Non-optimized objects (no defaults entry for their group value):

1. Skipped entirely - no modifications.
2. All original properties preserved as-is.
3. Ready for rendering without merge.

After each loop the merger deletes the merged defaults (preloadAssetsDefaults, animationsDefaults) from the room data.

Why this matters

The merger resolves the group value the same way the server filter does, so both sides group and restore by the exact same value:

• Objects without a defaults entry: were NOT optimized by the server, have complete data, use the asset_key or key field as asset reference.
• Objects with a defaults entry: were optimized by the server, have partial data, need the defaults merged.

The merger never rewrites the key field: the filter never extracts key to the defaults, so every object keeps its own value.

Data Flow Examples

Town NPCs (no optimization)

Server processing:

// Original data, two of the town NPCs (objects 5 and 8, layer 'ground', tiles 3482 and 3567)
objectsAnimationsData: {
  'ground3482': {
    key: 'people_town_1',
    content: 'Hello! My name is Alfred...',
    ...all properties...
  },
  'ground3567': {
    key: 'healer_1',
    ...all properties...
  }
}

// SceneDataFilter analysis:
// - Group by 'asset_key', falling back to the 'key' field (GroupValueResolver), no asset_key present
// - Each object has unique 'key' value = single-object groups
// - No optimization performed for them

// Server output
{
  objectsAnimationsData: { ...unchanged... },
  // only the door_house_3 entry of the town doors, nothing for the NPCs
  animationsDefaults: {...}
}

Client processing:

// AnimationsDefaultsMerger.mergeDefaults() runs
for(let key of ['ground3482', 'ground3567']){
    let objectData = objectsAnimationsData[key];
    // resolved group value: 'people_town_1' / 'healer_1' (from the key field)
    let groupValue = GroupValueResolver.resolve(objectData, 'asset_key');
    // animationsDefaults has no entry for these group values:
    if('' === groupValue || !sc.hasOwn(animationsDefaults, groupValue)){
        // SKIP - no modifications, keep original data
        continue;
    }
}

// Result: the NPC objects are unchanged
objectsAnimationsData: {
  'ground3482': {key: 'people_town_1', ...},
  'ground3567': {key: 'healer_1', ...}
}

// AnimationEngine uses props.key fallback
// object['ground3482'].key = 'people_town_1' loads asset 'people_town_1' and the NPC dialog works

Forest room (with optimization)

Illustrative example, the object keys, the property values and the 400 objects are not the sample data ones.

Server processing:

// Original data: 400 objects, 200 identical enemies per type
objectsAnimationsData: {
  'enemy_1': {
    // Already set by server
    asset_key: 'enemy_forest_1',
    type: 'npc',
    enabled: true,
    targetName: 'enemy-pve',
    layerName: 'enemies-layer',
    x: 100,
    y: 200
  },
  'enemy_2': {
    asset_key: 'enemy_forest_1',
    type: 'npc',
    enabled: true,
    targetName: 'enemy-pve',
    layerName: 'enemies-layer',
    x: 150,
    y: 250
  },
  // ... 198 more with same asset_key
}

// SceneDataFilter analysis:
// - Group by 'asset_key' field
// - 'enemy_forest_1' group has 200 objects
// - Detects identical: type, enabled, targetName, layerName
// - Keeps unique: x, y (different per object)
// - Keeps grouping field: asset_key (needed for lookup)

// Server output
{
  objectsAnimationsData: {
    'enemy_1': {asset_key: 'enemy_forest_1', x: 100, y: 200},
    'enemy_2': {asset_key: 'enemy_forest_1', x: 150, y: 250},
    // ... 198 more (only unique props + asset_key)
  },
  animationsDefaults: {
    'enemy_forest_1': {
      type: 'npc',
      enabled: true,
      targetName: 'enemy-pve',
      layerName: 'enemies-layer',
      // ... all shared properties
    }
  }
}

Client processing:

// AnimationsDefaultsMerger.mergeDefaults() runs
for(let key of ['enemy_1', 'enemy_2', ...]){
    let objectData = objectsAnimationsData[key];
    // {asset_key: 'enemy_forest_1', x: 100, y: 200}

    // Resolve the group value: 'enemy_forest_1'
    let groupValue = GroupValueResolver.resolve(objectData, 'asset_key');
    if('' === groupValue || !sc.hasOwn(animationsDefaults, groupValue)){
        // NOT executed - the defaults entry exists
        continue;
    }

    // Merge
    objectsAnimationsData[key] = Object.assign({}, animationsDefaults[groupValue], objectData);
    // Result: {
    //   type: 'npc',
    //   enabled: true,
    //   targetName: 'enemy-pve',
    //   layerName: 'enemies-layer',
    //   asset_key: 'enemy_forest_1',
    //   x: 100,
    //   y: 200
    // }
}

// AnimationEngine uses props.asset_key (exists) and loads asset 'enemy_forest_1'
// All properties restored from defaults + unique props

Performance Impact (400 objects, forest room)

Illustrative estimate for a room with 400 objects, the sizes below were not measured on the sample data rooms.

Unfiltered:

• preloadAssets: 400 x 2 assets x 275 bytes = 220 KB
• objectsAnimationsData: 400 x 150 bytes = 60 KB
• roomData: 10 KB
• Total sceneData: ~290 KB
• Total State (with 50 players): ~176 KB encoded
• Buffer overflow: required 176 KB vs 8 KB default

Optimized:

• preloadAssets: 2 spritesheets x 95 bytes = 0.2 KB
• objectsAnimationsData: 400 x 35 bytes = 14 KB
• animationsDefaults: 2 entries x 140 bytes = 0.3 KB
• roomData: 10 KB
• Total sceneData: ~25 KB
• Total State (with 50 players): ~80 KB encoded

Reduction:

• sceneData: 265 KB saved (91% reduction)
• Total State: 96 KB saved (54.5% reduction)

Configuration

sendAll flag

• Path: server/rooms/data/sendAll (config table: scope server, path rooms/data/sendAll)
• Default: false (filtering enabled). The path is not seeded, the filter falls back to the default.

REPLACE INTO `config` (`scope`, `path`, `value`, `type`) VALUES
('server', 'rooms/data/sendAll', '0', 3);

Values (boolean config type, stored as 0 / 1):

• false: enables optimization (recommended for production).
• true: sends all data unfiltered (debugging only).

Custom processor

For custom filtering logic, define a processor in your server plugin.

• Path: server/customClasses/sceneDataProcessor
• Method: process({ roomData, filter })

class CustomSceneDataProcessor {
    process({ roomData, filter }) {
        let customData = Object.assign({}, roomData);
        // Use filter methods for standard optimization
        let optimized = filter.buildFilteredData(roomData);
        // Add custom fields
        customData.customField = 'custom value';
        return Object.assign({}, optimized, customData);
    }
}

// on the ServerManager config object (theme/plugins/server-plugin.js):
customClasses: {
    sceneDataProcessor: new CustomSceneDataProcessor()
}

Key Concepts

asset_key field

Purpose: reference to the shared defaults, used for grouping and lookup.

When present:

• Set on the object data built by the server (the filter never adds it).
• Used as the grouping value, so the object may have partial data and need the defaults merged.
• The client uses it to look up the defaults and as asset reference.

When NOT present: the client uses the key field as asset reference, and the merger falls back to key as grouping value.

key field (two different roles)

1. Objects without asset_key: asset reference (e.g. people_town_1). Original value from the server. AnimationEngine fallback: sc.get(props, 'asset_key', props.key). Used to load the sprite asset and as grouping value fallback.
2. Objects with asset_key: object identifier. Original value from the server, never rewritten by the client merger. Not used for asset loading (asset_key is used instead).

Grouping fields

Purpose: the field used to group objects for comparison and defaults lookup.

Requirements:

• Must be identical across all objects in the group.
• Must stay in each object (NOT extracted to the defaults).
• The client needs it to look up the correct defaults entry.

Examples:

• asset_type for preloadAssets.
• asset_key for objectsAnimationsData (if present).
• key for objectsAnimationsData (fallback if there is no asset_key).

Why the grouping field must stay in the objects

// If asset_key was extracted to defaults:
objectsAnimationsData: {
  // No asset_key!
  'enemy_1': {x: 100, y: 200}
}
animationsDefaults: {
  'enemy_forest_1': {asset_key: 'enemy_forest_1', type: 'npc', ...}
}

// Client can't merge - doesn't know which defaults to use!
// No way to know 'enemy_1' should use 'enemy_forest_1' defaults

Keeping the grouping field in each object allows the lookup:

// 'enemy_forest_1'
let assetKey = objectData.asset_key;
// Found!
let defaults = animationsDefaults[assetKey];

Integration Points

Server integration

RoomScene.onCreate (lib/rooms/server/scene.js):

this.sceneDataFilter = new SceneDataFilter({config: this.config});
// room data is saved on the state:
let roomState = new State(this.roomData, this.sceneDataFilter);

State (lib/rooms/server/state.js), the constructor keeps the room data and the filter and calls mapRoomData():

this.roomData = roomData || {};
/** @type {SceneDataFilter|boolean} */
this.sceneDataFilter = sceneDataFilter || false;
this.mapRoomData();

State.mapRoomData(roomData) falls back to this.roomData when no data is passed and then filters it:

if(this.sceneDataFilter && this.sceneDataFilter.filterRoomData){
    roomData = this.sceneDataFilter.filterRoomData(roomData);
}
/** @type {string} */
this.sceneData = sc.toJsonString(roomData);

Client integration

RoomEvents.checkAndCreateScene() (lib/game/client/room-events.js):

if(0 === Object.keys(this.roomData).length){
    this.roomData = AnimationsDefaultsMerger.mergeDefaults(sc.toJson(this.room.state.sceneData));
}

AnimationEngine (lib/objects/client/animation-engine.js), the constructor (constructor(gameManager, props, currentPreloader)) uses asset_key if present and falls back to key:

this.asset_key = sc.get(props, 'asset_key', props.key);

Testing

Verify optimization behaviour

Town room (doors optimized, NPCs not):

1. Join the reldens-new-age-town room.
2. Verify: animationsDefaults only has the door_house_3 entry.
3. Check the browser console: the NPC objects keep all their data.
4. Test the doors open and the NPC dialogs work correctly.

Forest room (optimization expected):

1. Join the reldens-forest-level-1 room (12 Tree and 18 Tree Punch enemies and 10 mining rocks in the sample data).
2. Check the browser console: objects have the asset_key field.
3. Verify: animationsDefaults has entries.
4. Test the enemies render and behave correctly.

Measure data size

Add logging in State.mapRoomData():

this.sceneData = sc.toJsonString(roomData);
Logger.info('sceneData size: ' + this.sceneData.length + ' bytes');

Verify the buffer overflow is resolved

node theme/plugins/bot.js --numClients 50 --room reldens-bots-forest --endpoint http://localhost:8080

Expected: no buffer overflow warnings in the server console.

Verify client functionality

1. Join rooms with various object counts.
2. Verify the spritesheets load correctly.
3. Verify the animations play correctly.
4. Verify the NPC dialogs work correctly.
5. Check the browser console for errors related to missing assets or properties.

Debugging

Disable filtering

Set in the database (the value is read once, when the room creates the filter):

REPLACE INTO `config` (`scope`, `path`, `value`, `type`) VALUES
('server', 'rooms/data/sendAll', '1', 3);

Sends all database fields to the client for debugging. Compare filtered vs unfiltered data to identify issues.

Log optimization results

class DebugSceneDataProcessor {
    process({ roomData, filter }) {
        let filtered = filter.buildFilteredData(roomData);
        Logger.info('objectsAnimationsData count:', Object.keys(filtered.objectsAnimationsData || {}).length);
        Logger.info('animationsDefaults entries:', Object.keys(filtered.animationsDefaults || {}).length);

        // Log which objects were optimized
        for(let key of Object.keys(filtered.objectsAnimationsData || {})){
            let obj = filtered.objectsAnimationsData[key];
            if(sc.hasOwn(obj, 'asset_key')){
                Logger.info('Optimized object:', key, 'asset_key:', obj.asset_key);
            }
        }

        return filtered;
    }
}

Common issues

NPCs not visible:

• Check the browser console for asset loading errors.
• Verify the asset_key or key field is present in the object.
• Verify the asset exists in preloadAssets.
• Check AnimationEngine.asset_key is set correctly.

NPC dialogs not working:

• Verify non-optimized objects keep their original key field value.
• Check AnimationsDefaultsMerger is NOT modifying objects without a defaults entry for their group value.
• Verify the dialog system uses the correct object reference.

Buffer overflow still occurring:

• Verify sendAll: false in the config.
• Check the optimization is detecting shared properties.
• Log the data size before and after filtering.
• Verify the client is merging the defaults correctly.

References

• Server filter: lib/rooms/server/scene-data-filter.js
• Client merger: lib/game/client/animations-defaults-merger.js
• Group value resolver: lib/game/group-value-resolver.js
• State integration: lib/rooms/server/state.js
• Scene integration: lib/rooms/server/scene.js
• Room events: lib/game/client/room-events.js
• Animation engine: lib/objects/client/animation-engine.js
• Colyseus schema: https://docs.colyseus.io/state/schema/

Related Documentation

• Object Animations Engine - How client_key and asset_key are used on the client.
• Rooms Entity - Room configuration and tilemap setup.
• Enemy and Object Respawn Flow - Where objectsAnimationsData rows come from.
• Feature Modules - lib/rooms/ and lib/game/ internals.
• Configuration - General config system.