This is a set of scripts to automate the upload of song covers made by pop2piano directly to YouTube; an automatic pipeline, if you will.
http://sweetcocoa.github.io/pop2piano_samples
We start in one folder with all the songs we want.
The pipeline works by having the same filename, except for the extension, for the different parts (image, song, video, ...).
So for example, we start with artist - song name.mp3, generate artist - song name.png with Stable Diffusion, and so on.
For each project (pop2piano, Stable diffusion, ...) you are invited to go the linked project page for full installation instructions which are outside of the scope of this document. It only provides the glue to make all of them work together.
This pipeline can easily be adapted at each step, so make it your own!
This has been tested on Linux, but it would probably work on Windows with WSL or macOS. You need python (and ideally the ability to manage some virtual environments), ffmpeg, imagemagick, Stable Diffusion if you want to generate images. Pop2piano runs decently on CPU, SD needs a GPU.
pop2piano -c 2 -o outputs .
cd outputs
mkdir mixes
mv *.wav mixes
perl-rename 's/.dpipqxiy.composer2//' *
for f in *.mid; do timidity "$f" -Ow -o "${f%.*}".wav; done; To run pop2piano in local, I've made a forked branch: https://github.com/Le09/pop2piano/tree/install
To generate the wav from the midi, refer to: https://wiki.archlinux.org/title/Timidity++
I've used the Arachno soundfont.
This step is manual! After generating all url files, I manually write the Youtube link.
for f in *.wav; do echo "" > "${f%.*}".url; done; After this step our folder should contain song_name.url for each song_name.wav.
Automation of this step requires a Python script on top of the command line. It gives the option to input an alternate prompt for each song (some prompts ended up almost NSFW so this was needed). Because the generation of each image batch is fairly long, it makes a bell song at each iteration to help knowing when to input the name.
For the installation, refer to: https://github.com/AUTOMATIC1111/stable-diffusion-webui
Automation guide: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API
You need to launch it with the API argument, on Linux ./webui.sh --api.
import os
import sys
import base64
import requests
root_url = "http://127.0.0.1:7860/sdapi/v1/"
default_payload = {
"batch_size": 1,
"n_iter": 1,
"steps": 20,
"cfg_scale": 7,
"width": 512,
"height": 512,
"restore_faces": True,
}
def get_images(file_name, prompt, **kwargs):
if not os.path.isdir(file_name):
os.mkdir(file_name)
if prompt != file_name:
with open(os.path.join(file_name, prompt), 'w') as f:
f.write("\n")
data = dict(default_payload, prompt=prompt, **kwargs)
r = requests.post(root_url + "txt2img", json=data)
imgs_encoded = r.json()["images"]
for i, encoded in enumerate(imgs_encoded):
img_name = file_name + "_" + str(i) + ".png"
img_path = os.path.join(file_name, img_name)
with open(img_path, 'wb') as f:
f.write(base64.b64decode(encoded))
if __name__ == '__main__':
file_path = sys.argv[1]
file_name = os.path.splitext(file_path)[0]
prompt = input("Put alternate prompt for cover images (nothing to use the title):\n")
if not prompt:
prompt = file_name
get_images(file_name, prompt)Now we just need to loop over each song and build the composite:
mkdir -p imgs
for f in *.wav; do
ffolder="${f%.*}"
echo $ffolder
mkdir -p "$ffolder"
python sd.py "$f"
montage -mode Concatenate -tile 2x "$ffolder"/*.png imgs/"$ffolder".png
paplay /usr/share/sounds/freedesktop/stereo/complete.oga
doneAt this step our folder should contain imgs/song_name.png for each song_name.wav.
Each image in imgs should be 1024x1024 (since it is made of four 512 pixels square images).
Then generate manually the common left hand side of the video image.
It is called pop2pianothumb.png file, and should be 896x1080 to end up with a 1920x1080 image.
This could be optional as long at after this step each song_name.wav is paired with a
song_name.png (so in the same folder, not in the imgs subfolder anymore).
mkdir -p cover_imgs
for f in imgs/*.png; do
montage -mode Concatenate -tile 2 -gravity center pop2pianothumb.png "$f" -background black cover_"$f"
done
mv cover_imgs/* .
rm cover_imgsMore information on ImageMagick here: https://wiki.archlinux.org/title/ImageMagick
At this step our folder should contain song_name.png and song_name.wav.
Generating the videos is thus a simple ffmpeg step.
for f in *.wav; do
ffmpeg -loop 1 -i "${f%.*}".png -i "$f" -c:v libx264 -tune stillimage -c:a aac -b:a 192k -pix_fmt yuv420p -shortest "${f%.*}".mp4
doneMore information on FFmpeg here: https://wiki.archlinux.org/title/FFmpeg
At this step our folder should contain song_name.mp4 and song_name.url.
You just need to complete the description, input your YouTube secrets and the
file paths to the video files.
This upload is only barely changed from the example: https://github.com/pillargg/youtube-upload
Refer to this for a guide on how get access to the YouTube API, navigate the Google console to create the secrets.
import os
from youtube_upload.client import YoutubeUploader
secret_path = "/path/to/client_secret.ytupload_desktop.googleusercontent.json"
root_folder = "/path/to/Music/to_cover"
description = """Link to the original song: %s
The rest of your video description.
"""
def upload_video(root, filepaths):
uploader = YoutubeUploader(secrets_file_path=secret_path)
uploader.authenticate()
for file_name in filepaths:
file_path = os.path.join(root, file_name)
name = file_name.replace(".mp4", "")
url = open(file_path.replace("mp4", "url"), "r").read().strip()
title = f"{name} (pop2piano AI cover)"
options = {
"title" : title,
"description" : description % url,
"tags" : ["ai", "pop2piano", "piano cover"],
"categoryId" : "10",
"privacyStatus" : "private", # Video privacy. Can either be "public", "private", or "unlisted"
"kids" : False,
}
uploader.upload(file_path, options)
uploader.close()
if __name__ == '__main__':
file_paths = []
for file_path in os.listdir(root_folder):
if file_path.endswith(".mp4"):
file_paths.append(file_path)
upload_video(root_folder, file_paths)After a test you can directly make the videos public :-)
Thx for sharing this. Very cool! 🙏