C++ Gaming: World-state snapshot example for non-interrupting saves (8 threads, 3 million vars)

SaveStateExample_Large.cpp

// SaveStateExample_Large.cpp (C++11)
// AUTHOR: xixas | DATE: 2022.01.02 | LICENSE: WTFPL/PDM/CC0... your choice
// DESCRIPTION: Gaming: World-state snapshot example for non-interrupting saves.
// BUILD: g++ -o save-state-example-large SaveStateExample_Large.cpp -lpthread -std=c++11

#include <cstdio> // printf
#include <atomic> // std::atomic
#include <mutex>  // std::mutex
#include <thread> // std::thread

std::atomic<bool> isSaving(false); // Thread-safe save state flag

struct Updateable { virtual void update() const = 0; };

// Generic type wrapper with snapshot capability.
// Temporarily allocates memory for a second value if updated during save.
template <typename T> class Saveable : public virtual Updateable {
protected:
    mutable T *a, *b;

public:
    Saveable() { a = b = new T; }
    Saveable(const T& t) : Saveable() { *a = t; }
    Saveable(const Saveable<T>& o) : Saveable() { *a = *(o.a); if (o.a != o.b) { b = new T; *b = *(o.b); }}
    ~Saveable() { if (a != b) { delete a; } delete b; }

    Saveable<T>& operator=(const T& t) { setValue(t); return *this; } // Direct assignment
    operator T() { return getValue(); }                               // Direct use as type
    T operator +() { return getSaveValue(); }                         // Retrieve save value with a '+' prefix

    void setValue(const T& t) { if (isSaving && a == b) { b = new T; } *b = t; }
    T getValue() const { return *b; }
    T getSaveValue() const { return *a; }

    // Must call when full save is complete!
    void update() const override { if (a != b && !isSaving) { delete a; a = b; }}
};
typedef Saveable<bool>  bool_s;
typedef Saveable<int>   int_s;
typedef Saveable<float> float_s;

//-[ Example ]----------------------------------------------------------------
const size_t varCount = 1000000, threadCount = 8, maxSet = 25, saveEvery = 4;
std::mutex   mtx;
bool_s       worldBools[varCount];
int_s        worldInts[varCount];
float_s      worldFloats[varCount];

void initVars() {
    for (size_t i = 0; i < varCount; ++i) { worldBools[i]  = false; }
    for (size_t i = 0; i < varCount; ++i) { worldInts[i]   = 0; }
    for (size_t i = 0; i < varCount; ++i) { worldFloats[i] = 0.0f; }
}

void updateVars() {
    for (size_t i = 0; i < varCount; ++i) {
        worldBools[i].update();
        worldInts[i].update();
        worldFloats[i].update();
    }
}

void beginSave() { isSaving = true; }
void endSave() { isSaving = false; updateVars(); }

void thread(int id, bool canInitSave, bool canPerformSave) {
    bool done = false;
    while (!done) {

        // Lock and update world values
        mtx.lock();
        if (!isSaving || !canPerformSave) {
            if (worldInts[0] >= maxSet) { mtx.unlock(); break; } // early exit
            for (size_t i = 0; i < varCount; ++i) {
                bool_s&  worldBool  = worldBools[i];
                int_s&   worldInt   = worldInts[i];
                float_s& worldFloat = worldFloats[i];
                worldBool  = !worldBool;
                worldInt   = worldInt + 1;
                worldFloat = worldFloat + 0.5f;
            }
        }
        const bool  b = worldBools[0],  b_ = +worldBools[0];
        const int   n = worldInts[0],   n_ = +worldInts[0];
        const float f = worldFloats[0], f_ = +worldFloats[0];
        int state = !isSaving ? 1 : !canPerformSave ? 2 : 3;
        bool initSave = canInitSave && !isSaving && n && !(n % saveEvery);
        if (initSave) { beginSave(); }
        if (state == 3) { endSave(); }
        done = worldInts[0] >= maxSet && (!isSaving || !canPerformSave);
        mtx.unlock();

        // Display thread status and world state
        switch (state) {
            case 1:  printf("thread %d: Active: [%d, %d, %.1f]\n", id, b, n, f); break;
            case 2:  printf("thread %d: Active: [%d, %d, %.1f] -> Snapshot: [%d, %d, %.1f]\n", id, b, n, f, b_, n_, f_); break;
            default: printf("thread %d: Save Complete... [%d, %d, %.1f]\n", id, b_, n_, f_); break;
        }
        if (initSave) { printf("Thread %d: Saving...\n", id); }
    }
};

int main() {
    setlocale(LC_ALL, "");
    printf("Variable Count: %'u\n", varCount * 3);

    initVars();

    // Spawn threads
    std::thread threads[threadCount];
    for (size_t i = 0; i < threadCount; ++i) { threads[i] = std::thread(thread, i+1, true, i+1 == threadCount); }
    for (size_t i = 0; i < threadCount; ++i) { threads[i].join(); }
    return 0;
}

Quick saving a larger dataset

This example briefly expands on some of the concepts in an earlier example here to better evaluate performance at scale.

Basic variable registration is inherently incorporated (since it's using pre-sized arrays) and, as each thread is locking the dataset for its read/write block, I've removed the unnecessary ad-hoc locking in the class itself, as well as the per-read save state check for better performance. This more accurately mirrors the frame-based nature of game development.

Also, timeouts have been removed to let the threads fight it out, which actually increases the save duration's number of intermediate interrupting threads.

Compiled with -O3 optimization for the sake of performance testing.

Sample Output (timed)

$ g++ -O3 -o save-state-example-large SaveStateExample_Large.cpp -lpthread -std=c++11 && time ./save-state-example-large

Variable Count: 3,000,000
thread 1: Active: [1, 1, 0.5]
thread 2: Active: [0, 2, 1.0]
thread 3: Active: [1, 3, 1.5]
thread 3: Active: [0, 4, 2.0]
Thread 3: Saving...
thread 5: Active: [1, 5, 2.5] -> Snapshot: [0, 4, 2.0]
thread 6: Active: [0, 6, 3.0] -> Snapshot: [0, 4, 2.0]
thread 6: Active: [1, 7, 3.5] -> Snapshot: [0, 4, 2.0]
thread 8: Save Complete... [0, 4, 2.0]
thread 8: Active: [0, 8, 4.0]
Thread 8: Saving...
thread 2: Active: [1, 9, 4.5] -> Snapshot: [0, 8, 4.0]
thread 4: Active: [0, 10, 5.0] -> Snapshot: [0, 8, 4.0]
thread 3: Active: [1, 11, 5.5] -> Snapshot: [0, 8, 4.0]
thread 5: Active: [0, 12, 6.0] -> Snapshot: [0, 8, 4.0]
thread 7: Active: [1, 13, 6.5] -> Snapshot: [0, 8, 4.0]
thread 6: Active: [0, 14, 7.0] -> Snapshot: [0, 8, 4.0]
thread 1: Active: [1, 15, 7.5] -> Snapshot: [0, 8, 4.0]
thread 8: Save Complete... [0, 8, 4.0]
thread 2: Active: [0, 16, 8.0]
Thread 2: Saving...
thread 4: Active: [1, 17, 8.5] -> Snapshot: [0, 16, 8.0]
thread 3: Active: [0, 18, 9.0] -> Snapshot: [0, 16, 8.0]
thread 5: Active: [1, 19, 9.5] -> Snapshot: [0, 16, 8.0]
thread 7: Active: [0, 20, 10.0] -> Snapshot: [0, 16, 8.0]
thread 6: Active: [1, 21, 10.5] -> Snapshot: [0, 16, 8.0]
thread 1: Active: [0, 22, 11.0] -> Snapshot: [0, 16, 8.0]
thread 8: Save Complete... [0, 16, 8.0]
thread 2: Active: [1, 23, 11.5]
thread 4: Active: [0, 24, 12.0]
Thread 4: Saving...
thread 3: Active: [1, 25, 12.5] -> Snapshot: [0, 24, 12.0]
thread 8: Save Complete... [0, 24, 12.0]

real	0m1.408s
user	0m1.150s
sys	0m0.258s

Sample Conditions

This example's been set to a default of 8 threads and 3 million data objects (varCount of 1_000_000 times the 3 sample variable types).

Assumes worst case scenario - all values are updated by each thread on every iteration.

Test Setup:

Sampled on a i7-3930K (circa Q4 2011) (6 cores @ 3.2GHz)
System was under standard use -- few browser tabs open, music playing, etc.

Performance

1.15 seconds of user time over 25 iterations
or 0.046 seconds per iteration -- slightly less if you account for initialization.

However, commenting out the display block, so we're just looking at data times the results improve significantly:
Average of 0.640 user time (over 10 runs)
or 0.0256 seconds per iteration

Consequence:

You wouldn't want to do this (update millions of values on every frame) if you were shooting for a modern framerate of 60 fps.
The data times alone would near 2x your target framerate (0.016s at 60fps).

Then again, that's the nature of separation of concerns. Every thread shouldn't be getting an exclusive lock or updating every value. And, in practice, you don't access every value every frame -- just the ones that relate to the immediate player world state. Other values are generally batch processed on a separate thread, for example once a second, and then interpolated as necessary to maintain a semblance of a living world beyond that of the player's current perspective. This is doubly true in a multiplayer environment, where network latency (potentially hundreds of milliseconds) must be accounted for.