Skip to content

Instantly share code, notes, and snippets.

@mosra

mosra/anim.md Secret

Last active July 28, 2016 21:27
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save mosra/0ff8443d623cc0b09efb7a7299a0f692 to your computer and use it in GitHub Desktop.
Save mosra/0ff8443d623cc0b09efb7a7299a0f692 to your computer and use it in GitHub Desktop.
Animation braindump

Animation support

Low-level support

Bezier curve segment

I had a prototype lying around for quite some time but then ran git clean -f while tired and all is lost :D Basically:

namespace Magnum {

namespace Math {
    template<UnsignedInt order, UnsignedInt dimensions, class T> class Bezier {
        public:
            //...

            std::array<Bezier<order, dimensions, T>, 2> subdivide(Float t) const;

            Vector<dimensions, T> lerp(Float t) const;

        private:
            Vector<dimensions, T> _points[order + 1];
    };
    template<UnsignedInt dimensions, class T> using QuadraticBezier = Bezier<2, dimensions, T>;
    template<UnsignedInt dimensions, class T> using CubicBezier = Bezier<3, dimensions, T>;
    template<class T> using QuadraticBezier2D = QuadraticBezier<2, T>;
    template<class T> using QuadraticBezier3D = QuadraticBezier<3, T>;
    template<class T> using CubicBezier2D = CubicBezier<2, T>;
    template<class T> using CubicBezier3D = CubicBezier<3, T>;
}

typedef QuadraticBezier2D<Float> QuadraticBezier2D;
typedef QuadraticBezier2D<Double> QuadraticBezier2Dd;
//...

}

The subdivide() and lerp() shared some code (lerp at given t gives end point of first subdivided part and start point of the second) and the interpolation was done recursively using De Casteljau's algorithm. It was pretty fun to implement, but I don't remember the details anymore.

Keyframe

A stateless pair of timestamp and value corresponding to that timestamp, nothing else. Can be even

template<class T> using Keyframe = std::pair<Float, T>;

Because that's all you need. The keyframes can be then stored in any continuous memory (std::vector, std::array, Containers::Array, memory-mapped file...) and are constant and stateless.

The T can be anything that makes sense and can be interpolated between keyframes (Float, Vector3, bool, Quaternion, CubicBezier2D...).

Things to think about:

  • I used Float for representing seconds because it's the easiest type to calculate with, however the precision gets bad once you have big time values (days). Could be a problem. Useful reading: Don't store that in a float and a bazillion other articles on Bruce Dawson's blog. Another thing to consider might be using normalized timestamps in a range [0; 1] and then scaling them using appropriate type at runtime. Advantage is that the precision is always guaranteed, but that adds complexity and indirection.
  • There two ways to store the keyframes, based on what's the type being interpolated:
    • Either the above time-value pairs (where for 5 keyframes you have 5 values) and interpolation is done by taking two nearest keyframes and interpolating them using e.g. Math::lerp(a, b, t).
    • Or having the values "in between" -- for example a cubic Bezier segment is defined by begin/end point (position on previous and next keyframe) and two control points. In this case for 5 keyframes you would have 4 values. The interpolation is then done on a single segment, so e.g. a.lerp(t) Ideal would be to combine these two together somehow, but that would always mean one of the two cases would result in excessive data duplication. The other option is to maintain two independent interpolation implementations, one "pair-wise" and one "in-between".

We can always overengineer and have operator< and other stuff for keyframes, but I don't see a real use for that now.

Animation clip

template<class T> class AnimationClip {
    public:
        typedef ?? Interpolator;

    private:
        Containers::ArrayView<const Keyframe<T>> _keyframes;

        Float _beginTime, _endTime;
        Interpolator _interpolator;
}

Stateless view of a keyframe range with some metadata:

  • Containers::ArrayView<const Keyframe<T>> -- keyframe range
  • Begin time -- defaults to timestamp of first keyframe but you are able to move it earlier to extrapolate the first frame
  • End time -- defaults to timestamp of last keyframe but you are albe to move
  • Some "recipe" how to interpolate keyframes

Interpolators

In my case we handled only the "pair-wise" case and so the clip had just a pointer to a stateless function with signature

T f(const T&, const T&, Float)

Which fits basically any Math::*lerp() overload. In practice we had just two of them for the UI:

  • constant -- basically selecting nearest not later keyframe and returning its value. Ideal for animating integers, booleans etc.
  • linear -- Math::lerp()

I can think of more:

  • bezier -- which would need different function signature as it operates on single segment
  • some presets based on beziers (ease in, ease out, ...) -- example: http://easings.net/
  • spline... -- which would take more than just two keyframes to smooth out the curve
  • that thing used in OpenGEX

Things to think about:

  • How to handle different interpolator signature for the "in-between" keyframes and for more complex interpolators? For example I can smooth out Vector3s with Math::lerp() or using some complex iterative moothing function that takes five inputs.
  • Having a function pointer will prevent the call from being inlined so it will always be an indirection, costly if you are doing many animations in the frame. On the other hand, having the interpolator as part of the type (template parameter) would make it too templatey. Another option is to not specify the interpolator for the clip, but e.g in the Animator.

High-level classes

Animator

template<class T> class Animator {
    public:
        // ...

        enum class State {
            Stopped = 0,
            Playing,
            Paused
        };

        State state() const { return _state; }

        Int playCount() const { return _playCount; }
        Animator& setPlayCount(Int count) {
            _playCount = count;
            return *this;
        }

        Animator& play(Float time) {
            _state = State::Playing;
            _startTime = time;
            return *this;
        }
        Animator& pause(Float time) {
            if(_state != State::Paused) {
                _state = State::Paused;
                _pauseTime = time;
            }
            return *this;
        }
        Animator& stop() {
            _state = State::Stopped;
            _startTime = 0.0f;
            return *this;
        }

        T advance(Float time);

    private:
        const AnimationClip<T>& _clip;
        State _state{};
        Float _startTime{}, _pauseTime{};
        Int _playCount = 1;
        std::size_t _lastKeyframe{};
};

First stateful class. Contains a reference to an animation clip and current animation state. "User-facing" docs:

  • setPlayCount() -- sets play count. Default is play once then stop, setting to 0 will cause the animation to repeat indefinitely.
  • play() -- start the animation at given time or unpause paused animation. Setting the time to something else than current time is useful for syncing more animations together. The animation will play as long as play count allows, then transitions to Stopped state. Calling play() again while the animation is playing will rewind the animation from the beginning, resetting play count.
  • pause() -- pause the animation. Can be resumed by clicking play() again (won't reset play count), calling stop() stops the animation so play() will start again from the beginning. No-op if the animation is already paused.
  • stop() -- stops the animation, No-op if the animation is already stopped.

All the fun details are in advance(), which, for the user, just returns proper value for given time, but internally does all the magic. In case the animation is running, the following is done:

  1. Calculates current clip time as a modulo of the time passed to advance() function and clip time (diff between begin and end time)
  2. Checks time of frame at _lastKeyframe. If the time is larger than current clip time, _lastKeyframe gets reset to 0. This variable acts as a "hint" for avoiding searching linearly from the beginning every time.
  3. Then it goes through keyframe range starting from _lastKeyframe until it finds two nearest keyframes (don't forget to special case the extrapolating of first and last frame -- e.g. when extrapolating before first keyframe you select frames with index 0 and 1 and extrapolate with negative t).
  4. It converts the time to the t value for interpolation (use Math::lerpInverted()) and calls the interpolator function with given keyframe values.
  5. It saves the index of the first keyframe to _lastKeyframe and returns the interpolated value.

State handling (in order):

  1. In case the animation is running and the time passed to advance() exceeds play count, the state is set to stopped with _startTime kept as is.
  2. In case the animation is running but _pauseTime is not zero, _startTime is enlarged by the difference between _pauseTime and time passed to advance() so the animation is able to seamlessly unpause. _pauseTime is then reset back to zero.
  3. In case the animation is stopped and _startTime is zero (i.e. not yet started/stopped manually), it returns value from the first two keyframes extrapolated to beginning time.
  4. In case the animation is stopped and _startTime is not zero (i.e. stopped after exceeding play count), it returns value from the last two keyframes extrapolated to ending time.

The workflow with animators was basically having an instance of the Animator somewhere in your scene object and calling advance() on it every frame using some global timer value, then using the returned value to do the actual animation.

ConnectedAnimator

Handling the advance() values manually was sometimes a pain, so we introduced a version of the animator that was connectible using signals/slots. Internals were basically the same as in the "raw" Animator, but in addition every state and value change emitted a signal. We used Qt signals, here the Corrade::Interconnect library can achieve the same even without the ugly macro trickery (and without working around the inability to have templated classes with signals):

template<class T> class ConnectedAnimator: public Interconnect::Emitter {
    public:
        // ...

        Signal stateChanged(State previous, State current);
        Signal valueChanged(const T& value);
};

It was a separate class because the signals added quite some overhead and in many cases they weren't needed.

Things to think about:

  • Either have a completely separate class or have ConnectedAnimator derived from Animator so both can be used with the same APIs. That however means having the play()/pause()/stop()/advance() methods virtual which also adds overhead and is ugly.
  • More signals / signal specializations like paused()/unpaused()/stopped()/ started()...

Animator group

Basically a way to control a bunch of animators at once. In my case it was a vector of references to Animator instances. Functions:

  • play()/stop()/pause() -- just calling the equivalents on all the animators in the group.
  • advance() -- just calls Animator::advance() and ignores the return value. This was used only for firing signals from ConnectedAnimator instances, otherwise it has no point.

More

We wanted to be able to parametrize the animation values so there was another Animator derivative that could be parametrized with some transformation for the resulting value (for example we had a progress bar animation clip that went from 0.0f to 1.0f and we wanted the progressbar move from 200px to 700px, so we scaled the animation 500 times and translated on X by +200. This could be of course also done at each place where advance() is called, but this way it was way more convenient.

Scaling/parametrizing the time is also another option.

Also, all of the above counts with linear time. OpenGEX however doesn't assume linear time (and I fear Blender exports it non-linearly), so you have to dig into it and figure out how to do it ... I imagine it involves calculating t back from position on some CubicBezier1D curve, but haven't dug deeper, too scary at that time :D

@Squareys
Copy link

First of all: Thank you for taking time to do this! This is very important to be, a non-animated virtual world has pretty different quality than an animated one after all ;)

I added my notes below, the cancled out lines can be safely ignored, they just document the thought process how I realized I was wrong while writing ;)

There is only one thing I have a strong slightly different opinion about, see the biggest block of text below.

I used Float for representing seconds

How about a new template <class T> class Time: Unit<Time, T> class with _ms, _s, _m, _h literals? One could also type the Bezier/Keyframes/time-value pair to Time or Timef respectively:

template<class K, class V> using Keyframe = std::pair<K, V>;

I used Float for representing seconds (again... different, and imo more superior/flexible approach: )

Why not frames? 😉 Blender uses frames and an animation speed. You could therefore divide a second into 60 frames for example and then you would not have to deal with any precision issues. Only limitation is that you have this predefined grid of where keyframes can be set in time, which in my opinion does not matter, since it will be sampled down by your rendering frame rate anyway and even if you find an application that exports it, that would be a super rare use case. (solved by the following paragraph)
Also, you may want to speed up/slow down the animation, which I find works well with the mental model for "animation has speed in frames/second (always double?), frame count (T) and last keyframe (T)" rather than scaling up time.
You could still type it and have it float or double, but the Unit would be "frame" rather than time. This should allow for more flexibility (Use char for tiny animations? :P )
Updates would still happen using time, but according to the speed in fps.

Also: "frames" vs "keyframes", as in frames is Unit and "keyframe" is one of these unit values which is "key" to the animation => has a value associated with it.

setPlayCount()

Naming wise: setRepeatCount maybe? Could be more intuitive. Audio has setLooping but no loopCount or such, so consistency doesn't solve this for use, I guess. Not too important, though, playCount also makes sense in a way, once you see the doc for it.

it returns value from the last two keyframes extrapolated to beginning/ending time.

Did I understand correctly that ending/beginning time does not have a value associated with it? There could always be a keyframe which is earlier than the beginning time, so extrapolation would be interpolation again. To keep this consistent, I would define "no keyframe before/after begin/end time" as "default constructed value". I think that is how blender handles this also.

advance() on it every frame using some global timer value

How about a timediff? Possibly also typed with frames as above (again, note that a frame may be 1.02351614512341354 or whatever, if typed as double. If the speed is then set to 1.0 fps frame would be equivalent to time.

Handling the advance() values manually was sometimes a pain

Very crazy idea: What if Animation were to be a subclass of the value its typed on and sets itself by advance? This could allow using it as the value directly (with restrictions). I might be mixing up languages here though. Would work in Java, in C++ probably not for non-reference/pointer values, though. Also this doesn't handle cases where you need to set uniforms for example as a result of animation update.

How would you expect would virtual calls compare to signals performance-wise? You could provide stateChanged/valueChanged to override?
Or Events and/or Listeners (i.e. catching events or as a list in the animation to call state/value changed)? :)
Can you come up with a use case in which a animation triggers more than one update, which should not be done in the same Receiver/Listener/override?

We wanted to be able to parametrize the animation values

One could theoretically do everything in these derivatives (see overrides instead of signals).
I would rather put this outside of animations in the code handling the updates (on state change etc.).

@wivlaro
Copy link

wivlaro commented Jul 28, 2016

Any thoughts about animation blending too?

@wivlaro
Copy link

wivlaro commented Jul 28, 2016

I'm going to post this here too. This gives a good overview on the problem and a solution. http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html

@wivlaro
Copy link

wivlaro commented Jul 28, 2016

I also want to ask how tightly you're thinking of coupling this to the SceneGraph system? It seems a natural fit, but it's not strictly necessary. Usually we just need to generate a tree of transforms per-mesh (or sometimes shared, I guess). Having them separate might lend itself to some optimizations.

Eg. There's an option in Unity to not create whole a gameobject/transform tree (scenegraph objects). From Unity documentation:

Optimize Game Object - When turned on, the game object transform hierarchy of the imported character will be removed and stored in the Avatar and Animator component. The SkinnedMeshRenderers of the character will then directly use the Mecanim internal skeleton. This option improves the performance of the animated characters. You should turn it on for the final product. In optimized mode skinned mesh matrix extraction is also multithreaded.
https://docs.unity3d.com/Manual/FBXImporter-Rig.html

It should still be possible to get the transform of a rig node though, to be able to attach vfx or other things to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment