Example use of spleeter (Python+Tensorflow audio-extraction library and ffmpeg

spleeter-and-ffmpeg-quick-tutorial.md

This is a guide that basically combines protolium's very helpful ffmpeg cheatsheet with the spleeter library.

Here's a tweet thread that shows a video snippet, with separate bass, vocals, and drums track:

https://twitter.com/dancow/status/1191068710968209408

Installing stuff

ffmpeg

(this guide was tested on ffmpeg 4.1.4)

Lots of ways to do this. If you're on a Mac, I suggest using Homebrew:

brew install ffmpeg

spleeter

Repo: https://github.com/deezer/spleeter

HN discussion: https://news.ycombinator.com/item?id=21431071

Install (and isolate) using conda:

git clone https://github.com/Deezer/spleeter
conda env create -f spleeter/conda/spleeter-cpu.yaml
conda activate spleeter-cpu

You can also do pip install spleeter but you might run into more than a few library conflicts with your existing Python environment...

Extracting the audio from video

(From this helpful gist: https://gist.github.com/protrolium/e0dbd4bb0f1a396fcb55)

Given a video.mp4 file, here's how to use ffmpeg to extract the audio into a separate file, audio.wav:

ffmpeg -i video.mp4 -vn -f wav audio.wav

Splitting the audio into 5 tracks with spleeter

(From spleeter's Quick Start info: https://github.com/deezer/spleeter#quick-start)

The following command invokes spleeter to split audio.wav into 5 separate tracks, into a subdirectory called myoutput/audio

spleeter separate -i audio.wav -p spleeter:5stems -o myoutput

The generated files:

myoutput/audio/
├── bass.wav
├── drums.wav
├── other.wav
├── piano.wav
└── vocals.wav

Creating a video file with one of the extracted tracks

Basically, create a silent version of the video, and then merge it with one of the given track extracts.

Strip audio from video

https://gist.github.com/protrolium/e0dbd4bb0f1a396fcb55#replace-audio-on-a-video-without-re-encoding

This creates a copy of the video, sans audio track, named silent-video.mp4 from video.mp4

ffmpeg -i video.mp4 -codec copy -an silent-video.mp4

Merge silent video with audio track

https://gist.github.com/protrolium/e0dbd4bb0f1a396fcb55#replace-audio-on-a-video-without-re-encoding

Here's an example that creates drums-video.mp4: the video, with just the drums track:

ffmpeg -i silent-video.mp4 -i myoutput/audio/drums.wav -c:v copy -c:a aac drums-video.mp4

Make an excerpt of a media file (optional)

(StackOverflow: https://superuser.com/questions/377343/cut-part-from-video-file-from-start-position-to-end-position-with-ffmpeg)

If you want to snippet any media file, here's an example using ffmpeg that cuts a 12-second snippet, beginning at the 30th second:

ffmpeg -i video.mp4 -ss 30 -t 12  video-snippet.mp4

Can you help me create a bat file (ffmpeg) to extract vocals of audio using spleeter library?
Thank you so much.