Skip to content

Instantly share code, notes, and snippets.

@i30817
Last active September 8, 2022 21:29
Show Gist options
  • Save i30817/ba37fbb2b3c6e34ff926ad833f465055 to your computer and use it in GitHub Desktop.
Save i30817/ba37fbb2b3c6e34ff926ad833f465055 to your computer and use it in GitHub Desktop.
create m3u for retroarch based on name
#!/usr/bin/env bash
me=$(basename "$0")
export me
function usage(){
cat >&2 <<ENDOFHELP
Usage: $me [-r|--relative] [-c|--children] [-h|--help] TARGET [DESTINATION]
$me examines rom media files in TARGET and subdirectories
and for each set of filenames that only differs from a media
identifier - (Disc #) and similar constructs - attempts to
write a ordered m3u in DESTINATION with either absolute or
optionally with relative paths to those media files.
$me tolerates different games in the same directory.
If DESTINATION is missing, the m3u files will be created
on the directory the rom media files were found in, unless
-c is turned on.
-h show this help
-r m3us will have relative paths to the media files
-c the TARGET will be treated as a source of top level
directory names to be placed on DESTINATION with the
m3us resulting from the search of each child directory
created inside the new directories, you can use this
to create shallow top-level copy of dirnames with m3us
If no DESTINATION is given, a error stops processing
If TARGET has no directories, no output is done
TARGET and DESTINATION are not searched for games
ENDOFHELP
}
export RELATIVE=1
export CHILDREN=1
#silent mode, h or r or - followed by a 'argument' to handle long options
#(notice that all of these require - already, including '-')
while getopts ":hrc-:" opt; do
case $opt in
-)
case "${OPTARG}" in
relative) RELATIVE=0; ;;
children) CHILDREN=0; ;;
help) usage; exit 0; ;;
*) usage; exit 1; ;;
esac;;
r) RELATIVE=0; ;;
c) CHILDREN=0; ;;
h) usage; exit 0; ;;
*) usage; exit 1; ;;
esac
done
shift $((OPTIND-1))
[[ "$#" -eq 0 ]] && { usage; exit 1; }
[[ "$#" -gt 2 ]] && { echo >&2 "$me: $# positional arguments not allowed"; exit 1; }
[[ ! -d "$PWD" ]] && { echo >&2 "$me: could not retrieve current directory"; exit 1; }
[[ -z "$2" ]] && [[ "$CHILDREN" -eq 0 ]] && { echo >&2 "$me: the children switch requires a DESTINATION"; exit 1; }
#make sure both positional args (both dirs, that may not exist) tolerate '-'
#at the start of their name by absolutizing when necessary
TARGET=$(realpath -- "$1")
[[ -d "$TARGET" ]] || { echo >&2 "$me: target directory to search for m3u compatible files does not exist"; exit 1; }
#function and array to turn strings containing english substrings one,two..., thirty-one or BOOT,A,B,...Z,SAVE,SYSTEM to numbers
function disc_string_to_number(){
#associative arrays can't be exported so dont even think of moving it out
declare -A numerals_alpha_map=( [thirty-one]=31 [thirty]=30 [twenty-nine]=29 [twenty-eight]=28 [twenty-seven]=27 [twenty-six]=26 [twenty-five]=25 [twenty-four]=24 [twenty-three]=23 [twenty-two]=22 [twenty-one]=21 [twenty]=20 [nineteen]=19 [eighteen]=18 [seventeen]=17 [sixteen]=16 [fifteen]=15 [fourteen]=14 [thirteen]=13 [twelve]=12 [eleven]=11 [ten]=10 [nine]=9 [eight]=8 [seven]=7 [six]=6 [five]=5 [four]=4 [three]=3 [two]=2 [one]=1 [zero]=0 [BOOT]=0 [A]=1 [B]=2 [C]=3 [D]=4 [E]=5 [F]=6 [G]=7 [H]=8 [I]=9 [J]=10 [K]=11 [L]=12 [M]=13 [N]=14 [O]=15 [P]=16 [Q]=17 [R]=18 [S]=19 [T]=20 [U]=21 [V]=22 [W]=23 [X]=24 [Y]=25 [Z]=26 [SAVE]=32 [SYSTEM]=33 )
local numerals_ordered=(thirty-one thirty twenty-nine twenty-eight twenty-seven twenty-six twenty-five twenty-four twenty-three twenty-two twenty-one twenty nineteen eighteen seventeen sixteen fifteen fourteen thirteen twelve eleven ten nine eight seven six five four three two one zero)
local special_ordered=(BOOT SAVE SYSTEM)
local alpha_ordered=(A B C D E F G H I J K L M N O P Q R S T U V W X Y Z)
local orig_nocasematch
orig_nocasematch=$(shopt -p nocasematch; true)
shopt -s nocasematch || true #turn on (case off)
for key in "${numerals_ordered[@]}"; do
if [[ "$1" == "$key" ]]; then
echo -n "${numerals_alpha_map[$key]}"
return
fi
done
for key in "${special_ordered[@]}"; do
if [[ "$1" == "$key" ]]; then
echo -n "${numerals_alpha_map[$key]}"
return
fi
done
shopt -u nocasematch || true #turn off (case on)
for key in "${alpha_ordered[@]}"; do
if [[ "$1" == "$key" ]]; then
echo -n "${numerals_alpha_map[$key]}"
return
fi
done
$orig_nocasematch #reset nocasematch
echo -n ""
}
export -f disc_string_to_number
#$1 is the filename without the path, always with extension
#function writes to 2 associative arrays file2gamename and file2discgroup, and they are to be used for sorting
function segmentname(){
#warning: sed doesn't have noncapturing groups so every nonliteral () counts, including the (disc|cd|disk) group
local ext="${1##*.}" #parse extension
local name="${1%.*}" #remove extension
local disc
local possible_number
#if there was a 'first cd group' then remove it and keep all the others
#while we could remove the rest of the line to 'fit more'
#(things that when you remove the disc group have different suffixes because of further metadata)
#that's prone to false positives if the user places different versions of the same game in the same dir
#(translations, hacks, actual versions etc). The cases where it matters are in the clear minority
#and usually because the dumper couldn't help themselves placing (System) after (Disk #).
#removing other redundant disc groups (some dumping projects think it's a good idea) is the best i can do
#find the first occurrence of the pattern
disc=$(echo -n "$name" | grep -oiE '\([^)(]*(disk|cd|disc)[^)(]*\)' | head -n 1)
if [[ -n "$disc" ]]; then
name=$(echo -n "$name" | sed -nE 's/\([^)(]*(disk|cd|disc)[^)(]*\)//pi' )
else #check for other brackets
disc=$(echo -n "$name" | grep -oiE '\[[^][]*(disk|cd|disc)[^][]*\]' | head -n 1)
if [[ -n "$disc" ]]; then
name=$(echo -n "$name" | sed -nE 's/\[[^][]*(disk|cd|disc)[^][]*\]//pi' )
fi
fi
#echo "$disc"
#echo "$name"
if [[ -n "$disc" ]]; then
#find the first number, note there are actual 'disc groups' with more than one, (Disc 1 of X), but we need only the 1rst
possible_number=$(echo -n "$disc" | grep -oiE '[[:digit:]]+' | head -n 1)
if [[ -n "$possible_number" ]]; then #exists and contains numbers
#remove leading 0's (to make the version sort not freak out when it sees 01 and 11 in the same game)
possible_number=$(echo -n "$possible_number" | sed -nE 's/0+//p')
if [[ -n "$possible_number" ]]; then
disc="$possible_number"
else #was only zero(s)
disc="0"
fi
else #exists and does not contain numbers
#string numbers may be actual english numerals and constructs like 'A,B,BOOT,SAVE,SYSTEM' that have a natural order etc
#remove brackets by space, then 'disc', then leading spaces if they exist, then get the first word (only). Works with 'DiscA' and similar.
possible_number=$(echo -n "$disc" | sed -nE -e 's/[][)(]/ /g' -e 's/(disk|cd|disc)//i' -e 's/^\s*//p' | cut -d ' ' -f 1 )
possible_number=$(disc_string_to_number "$possible_number")
if [[ -n "$possible_number" ]]; then #replace numeral
disc="$possible_number"
else #give up and with no sort possible, reset to original text without the rest of the string cut
#warn the user one game has problems parsing
echo >&2 "$me: disc group $disc found but couldn't parse a number for it, try to use uppercase letters if you're using letters as names"
disc=""
name="${1%.*}"
fi
fi
fi
#echo "$disc"
#echo "$name"
#removing the cd group made a 'hole' which has two interesting cases, | (cd x) | and | (cd x).ext|
#it's unlikely that multiple dumps of the same game will differ in the number of spaces and be in the same dir
#so space manipulation so the final result looks good should be ok. Extension must be kept though
name=$(echo "$name" | sed -Ee "s/(^\s+|\s+$)//g" ) #trim leading and trailing spaces
name=$(echo "$name" | sed -Ee "s/\s\s*/ /g" ) #uniquify spaces
file2gamename["$1"]="$name.$ext"
file2discgroup["$1"]="$disc"
}
export -f segmentname
#$1 is the rom dir $2 is the m3u dir parent (which can be the current dir)
function create(){
shopt -s nocaseglob || true
shopt -s nullglob || true
shopt -s nocasematch || true
cd -- "$1" || { echo >&2 "$me: inaccessible directory: $1"; return; }
if [[ "$2" == "." ]]; then
FAKEM3U=1
for i in ./*.m3u; do
file -i "$i" | grep -qE ': text/plain; charset' || { FAKEM3U=0; echo >&2 "$me: $i is not a m3u text file."; }
done
if [[ "$FAKEM3U" -eq 0 ]]; then
echo >&2 "$me: refusing to delete unexpected files from directory $PWD"
return
fi
rm -f ./*.m3u || { echo >&2 "$me: couldn't delete the previous m3u files"; return; }
fi
#cue, toc and ccd are index files themselves and can point to isos (cue and toc) or img (ccd) so they have priority
names=(*.{cue,toc,ccd})
if [[ "${#names[@]}" -eq 0 ]]; then
names=(*.{mds,cdi,img,iso,chd,rvz,ipf,dim,adf,fdi,dms,adz,fds,qd})
fi
if [[ "${#names[@]}" -eq 0 ]]; then
return
fi
################################################################################################
# SORTING:
# assumed invariants
# each game with multiple media will have the same extension and case on their discs filenames
# each filename has no \n
# each filename only disc group either not exists or has one of the variants:
# (Disc x)|(Disc x of y)|[Disc x]|[Disc x of y]
# (Disk x)|(Disk x of y)|[Disk x]|[Disk x of y]
# (cd x)|(cd x of y)|[cd x]|[cd x of y]
#first create the associative arrays
#the call can't be in the 2nd loop because it's a subprocess
declare -A file2gamename
declare -A file2discgroup
for n in "${names[@]}"; do
segmentname "$n"
done
#(\0 never should hit bash vars and array expansion uses \n in mapfile)
#the V version sort supports putting 'disc 2' before 'disc 10' so it's what we want.
#but make sure to sort the disc group last, because there might be versions/metadata after
mapfile -t names < <(for key in "${names[@]}"; do
printf '%s\n%s%s\0' "$key" "${file2gamename[$key]}" "${file2discgroup[$key]}"
done | sort -zVt$'\n' -k2,2 | cut -zd$'\n' -f1 | tr '\0' '\n' )
################################################################################################
# M3U creation:
while [[ "${#names[@]}" -gt 0 ]] ; do
first="${names[0]}"
fname="${file2gamename[$first]}"
names=("${names[@]:1}") #removed the 1st element
gameset=("$first")
while [[ "${#names[@]}" -gt 0 ]] ; do
second="${names[0]}"
#if these are equal, they're part of the same game, if not it's a new game
if [[ "$fname" == "${file2gamename[$second]}" ]]; then
gameset+=("$second")
names=("${names[@]:1}")
continue
else
break
fi
done
#write the new m3u
m3u_name="${fname%.*}.m3u"
if [[ "$RELATIVE" -eq 0 ]]; then
cuedir="$(realpath --relative-to "$2" "$1")/"
#ommit the cue dir if the same dir and relative
if [[ "$cuedir" == "./" ]]; then
cuedir=""
fi
else
cuedir="$1/"
fi
printf '%s\n' "${gameset[@]/#/$cuedir}" > "$2/$m3u_name"
done
}
export -f create
function createaux(){
M3U_DIR="$(realpath -- "$2")"/"$(basename "$1")"
mkdir -p "$M3U_DIR" || { echo >&2 "$me: couldn't create the destination directory"; exit 1; }
shopt -s nullglob || true
FAKEM3U=1
for i in "$M3U_DIR"/*.m3u; do
file -i "$i" | grep -qE ': text/plain; charset' || { FAKEM3U=0; echo >&2 "$me: $i is not a m3u text file."; }
done
if [[ "$FAKEM3U" -eq 0 ]]; then
echo >&2 "$me: refusing to delete unexpected files from directory $M3U_DIR"
return
fi
rm -f "$M3U_DIR"/*.m3u || { echo >&2 "$me: couldn't delete the previous m3u files"; exit 1; }
LC_ALL=en_GB.UTF-8 find "$1" -type d -exec bash -c 'create "$1" "$2" ' -- "{}" "$M3U_DIR" \;
#if completely empty, delete the dir since it didn't produce m3us
if [ -z "$(ls "$M3U_DIR")" ]; then
rm -r "$M3U_DIR"
fi
}
export -f createaux
if [[ "$CHILDREN" -eq 0 ]]; then
LC_ALL=en_GB.UTF-8 find "$TARGET" -mindepth 1 -maxdepth 1 -type d ! -path "$2" -exec bash -c 'createaux "$1" "$2" ' -- "{}" "$2" \;
else
if [[ -z "$2" ]]; then
#without a destination delete and write later on the rom dir (because we're saving m3u's on different dirs where we find games)
M3U_DIR="."
else
#with a destination, delete now instead of later and absolutize
M3U_DIR="$(realpath -- "$2")"
mkdir -p "$M3U_DIR" || { echo >&2 "$me: couldn't create the destination directory"; exit 1; }
shopt -s nullglob || true
FAKEM3U=1
for i in "$M3U_DIR"/*.m3u; do
file -i "$i" | grep -qE ': text/plain; charset' || { FAKEM3U=0; echo >&2 "$me: $i is not a m3u text file."; }
done
if [[ "$FAKEM3U" -eq 0 ]]; then
echo >&2 "$me: refusing to delete unexpected files from directory $M3U_DIR"
return
fi
rm -f "$M3U_DIR"/*.m3u || { echo >&2 "$me: couldn't delete the previous m3u files"; exit 1; }
fi
#use utf-8 for this
LC_ALL=en_GB.UTF-8 find "$TARGET" -type d -exec bash -c 'create "$1" "$2" ' -- "{}" "$M3U_DIR" \;
fi
@birdybro
Copy link

Does this work on zip files contents or could it be made to do so if not?

I would like to create a ".m3u" zip sorta like redump.org's cue/gdi sheets with it and submit it to them, one for just the multi-disc games in their naming format. I have the entire redump set on a drive for just this sort of testing, but all zipped up :(

@i30817
Copy link
Author

i30817 commented Sep 20, 2020

But... redump cues can be downloaded as a single (much smaller zip) on their site, per platform. You wouldn't need anything else or to unzip the whole 'actual' set.

@i30817
Copy link
Author

i30817 commented Sep 7, 2022

This had a bug, where it would do the exact opposite of what i wanted when the disc group was not the last one.

ex: 'Langrisser IV & V - Final Edition (Japan) (Disc 2) (Langrisser V Disc)' was being turned into 'Langrisser IV & V - Final Edition (Japan)' instead of 'Langrisser IV & V - Final Edition (Japan) (Langrisser V Disc)'

This is probably fixed now. Bash is really a horrible language for this kind of stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment