Posy Motion Extraction

colordelay.sh

#!/bin/bash
# Usage: ./colordelay.sh input.mp4 2 output.mp4
# Delays the red, green and blue channels of a video by a specified number of frames
# The first argument is the path to the video to extract motion from
# The second, third and fourth arguments are the number of frames to delay the red, green and blue channels respectively (optional, defaults 0, 2, 4)
# The fifth argument is the output video (optional, default output.mp4)

input_video=$1
red_frames_offset=${2:-2}
green_frames_offset=${3:-2}
blue_frames_offset=${4:-2}
target_video=${5:-output.mp4}

temp_dir=$(mktemp -d)

# make input grayscale
ffmpeg -i $input_video -vf "format=gray" $temp_dir/grayscale.mp4

# split RGB channels into 3 separate grayscale videos
ffmpeg -i $temp_dir/grayscale.mp4 -filter_complex "[0:v]format=rgb24,split=3[r][g][b];[r]extractplanes=r[red];[g]extractplanes=g[green];[b]extractplanes=b[blue]" -map "[red]" $temp_dir/red.mp4 -map "[green]" $temp_dir/green.mp4 -map "[blue]" $temp_dir/blue.mp4

ffmpeg -i $temp_dir/red.mp4 -vf trim=start_frame=$red_frames_offset,setpts=PTS-STARTPTS $temp_dir/red_shifted.mp4
ffmpeg -i $temp_dir/green.mp4 -vf trim=start_frame=$green_frames_offset,setpts=PTS-STARTPTS $temp_dir/green_shifted.mp4
ffmpeg -i $temp_dir/blue.mp4 -vf trim=start_frame=$blue_frames_offset,setpts=PTS-STARTPTS $temp_dir/blue_shifted.mp4

# merge the shifted videos into GBRP format
ffmpeg -i $temp_dir/green_shifted.mp4 -i $temp_dir/blue_shifted.mp4 -i $temp_dir/red_shifted.mp4 -filter_complex "[0:v][1:v][2:v]mergeplanes=0x001020:gbrp" $temp_dir/gbrp_video.mp4

# convert the GBRP video to RGB format
ffmpeg -i $temp_dir/gbrp_video.mp4 -i $input_video -vf "format=rgb24" -c:v libx264 -preset ultrafast -c:a copy $target_video

motionextractor.sh

#!/bin/bash
# Usage: ./motionextractor.sh input.mp4 2 output.mp4
# Extracts motion from a video using the method described by Posy in https://www.youtube.com/watch?v=NSS6yAMZF78
# The first argument is the path to the video to extract motion from
# The second argument is the number of frames to shift the video by to detect motion (optional, default 2)
# The third argument is the output video (optional, default output.mp4)

input_video=$1
frames_offset=${2:-2}
target_video=${3:-output.mp4}
temp_dir=$(mktemp -d)
ffmpeg -i $input_video -vcodec png $temp_dir/input_%06d.png
ffmpeg -i $input_video -vf "negate,format=rgba,colorchannelmixer=aa=0.5" -vcodec png $temp_dir/inverted_halfopacity_%06d.png
ffmpeg -i $temp_dir/input_%06d.png -start_number $frames_offset -i $temp_dir/inverted_halfopacity_%06d.png -i $input_video -filter_complex "[0:v][1:v] overlay=0:0" -c:v libx264 -preset ultrafast -c:a copy $target_video

motionextractorfixed.sh

#!/bin/bash
# Usage: ./motionextractorfixed.sh input.mp4 1 output.mp4
# Extracts motion from a video using the method described by Posy in https://www.youtube.com/watch?v=NSS6yAMZF78
# The first argument is the path to the video to extract motion from
# The second argument is the fixed frame number to compare to to detect motion (1 is the first frame, not 0) (optional, default 1)
# The third argument is the output video (optional, default output.mp4)

input_video=$1
comparison_frame=${2:-1}
target_video=${3:-output.mp4}
temp_dir=$(mktemp -d)

# separate the input video into frames
ffmpeg -i $input_video -vcodec png $temp_dir/input_%06d.png
# create a video with the same frames but with half opacity and inverted colors
ffmpeg -i $input_video -vf "negate,format=rgba,colorchannelmixer=aa=0.5" -vcodec png $temp_dir/inverted_halfopacity_%06d.png
# overlay the inverted half opacity video on top of the fixed frame
ffmpeg -i $temp_dir/input_$(printf "%06d" $comparison_frame).png -i $temp_dir/inverted_halfopacity_%06d.png -i $input_video -filter_complex "[0:v][1:v] overlay=0:0" -c:v libx264 -preset ultrafast -c:a copy $target_video

PosyMotionExtraction.md

Posy Motion Extraction

This gist shows how to extract motion from a video using the method described by Posy in https://www.youtube.com/watch?v=NSS6yAMZF78

To get the motion from a video by comparing each frame to the one 5 frames later:

./motionextractor.sh input.mp4 5 output.mp4

To compare to the first frame:

./motionextractorfixed.sh input.mp4 1 output.mp4

To get the motion by making grayscale, splitting the RGB channels and separately delaying them by 0, 3 and 6 frames respectively:

./colordelay.sh input.mp4 0 3 6 output.mp4

A version of this is up on replicate at https://replicate.com/chigozienri/posy-motion-extraction, powered by https://github.com/chigozienri/cog-posy-motion-extraction