Skip to content

Instantly share code, notes, and snippets.

@eliranmal
Last active May 7, 2017 11:59
Show Gist options
  • Save eliranmal/6379b0908cea257a46318426f9eee390 to your computer and use it in GitHub Desktop.
Save eliranmal/6379b0908cea257a46318426f9eee390 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash
if [[ $HELP = true ]]; then
less << 'EOF'
overview
--------
migrates git-flux configurations from global to local context.
i.e. looks for any config entries (e.g. for branches) registered by git-flux in
the global git-config for a repository (or repositories), and moves them to a
repo's local git-config.
this is done as part of the git-flux context-awareness feature (see
https://github.com/eliranmal/git-flux/issues/33), in an effort to make git-flux
less error-prone for usage with multiple repositories.
usage
-----
curl -s https://.../git-flux-config-migration.sh | [environment] bash
there are two basic ways to invoke this script:
- inside a git repository (i.e. a directory with a nested '.git' directory),
that is found to be used with git-flux. in this mode, only that specific
repository will be affected.
- in your workspace directory (i.e. a directory that hosts repositories as
direct descendants). this mode will simply look up candidate repositories in
your workspace and activate the migration on each one.
check out the 'environment' section for more invocation options.
after you decided which mode you prefer (and what to put in the environment),
navigate to the relevant directory (repo/workspace) and run this:
curl -s https://gist.githubusercontent.com/eliranmal/6379b0908cea257a46318426f9eee390/raw/git-flux-config-migration.sh | bash
if something terrible happened, and you want to roll-back to how stuff were
before the execution, you can use the ROLLBACK env var (see below).
this script generates backup files for all git-config files (including the
global one), and places them near the originals (local git-configs - in each
repository's '.git' directory, and global - in the home directory). they are
suffixed with a numeric timestamp followed by the '.gitflux.bak' extension.
to get rid of them, use the PURGE env var (see below).
environment
-----------
you can use environment variables to do these stuff:
- HELP=[true|false]:
show this message, and exit. default is 'false'.
- DRY_RUN=[true|false]:
run the migration script without actually modifying the git-config files on
the system. instead, output the result to new config files, near the originals,
which will be suffixed with the .gitflux.dryrun extension. default is 'false'.
- PURGE=[backup|dryrun|all]:
delete config files generated during backup or dry-run phase, and
exit. default is 'false'.
- ROLLBACK=[true|false]:
restore git config files to a previous revision, and exit. default is 'false'.
EOF
exit 0
fi
main() {
log_title "- - - git-flux config migration - - -"
# prefix for the git-flux section
GIT_CONFIG_SECTION='gitflux'
# separator for branch/base associations
CONFIG_ENTRY_DELIMITER=':::'
# dry-run file extension
DRY_RUN_EXT='gitflux.dryrun'
# backup file extension
BACKUP_EXT='gitflux.bak'
# a timestamp constant for hashing backup files
# shellcheck disable=SC2034
BACKUP_UID="$(date "+%y%m%d%H%M%Y%S")"
# backup file extension
BACKUP_SUFFIX="$BACKUP_UID.$BACKUP_EXT"
if [[ $PURGE ]]; then
purge_artifacts; log
exit 0
fi
if [[ $ROLLBACK ]]; then
# restore a previous revision of config files
restore_backup; log
exit 0
fi
# temporary storage space for caching operations
TEMP_DIR="$(create_temp_dir)"
# handle generated files responsibly
trap 'on_exit' EXIT
if is_dry_run; then
# copy git config files for working without side-effects. this must be called before scan_repositories()
create_dry_run_sandbox
fi
# scan the filesystem for candidate repositories, and cache the results
scan_repositories
# backup git config files, including global, in case something happens
backup_configs
# move global git config entries into their candidate repositories
transfer_all_repos
# get rid of old global git config entries of candidate repositories
# this must be done after all migrations have completed,
# as there may be similar branch names between repositories
cleanup_all_repos
# get rid of any unmatched data (that's left in the global config, and was
# not picked up on any candidate repository)
cleanup_orphan_branch_entries
# get rid of global data that has been migrated already
cleanup_long_lived_branch_entries
}
# ----- high-level actions -----
create_dry_run_sandbox() {
local config_paths
# don't pass candidate-repos, we cannot filter them out of the entire repo
# list just yet, as this requires reading from the global git-config, which
# is not created for dry-runs at this point (creating it is the purpose of
# this function).
config_paths=$(list_config_paths)
if [[ -z $config_paths ]]; then
return 0
fi
log_title "[sandbox]"
copy_path "" ".$DRY_RUN_EXT" <<< "$config_paths"
}
backup_configs() {
local candidate_repo_dirs
local config_paths
local dryrun_suffix
log_title "[backup]"
candidate_repo_dirs="$(list_candidate_repositories)"
config_paths=$(list_config_paths <<< "$candidate_repo_dirs")
if is_dry_run; then
dryrun_suffix=".$DRY_RUN_EXT"
fi
copy_path "$dryrun_suffix" ".$BACKUP_SUFFIX" <<< "$config_paths"
}
scan_repositories() {
local status
local list
local count
log_title "[scan]"
if is_git_repo; then
log; log_ok "inside a git repository." 1
else
log_subtitle "outside of a git repository. searching for candidate repositories..." 1
list=$(list_candidate_repositories)
status=$?
count=$(count_lines <<< "$list")
if (( count < 1 )); then
log_fatal "no repositories found, aborting." 2
else
status_log_fatal $status "searched ok, $count repositories found:
$(print_lines 3 <<< "$list")" "search failed" 2
fi
fi
}
transfer_all_repos() {
local candidate_repo_dirs
candidate_repo_dirs=$(list_candidate_repositories)
while read -r dir; do
( cd "$dir" && transfer_repo ) || log_fatal
done <<< "$candidate_repo_dirs"
}
cleanup_all_repos() {
local candidate_repo_dirs
candidate_repo_dirs=$(list_candidate_repositories)
while read -r dir; do
( cd "$dir" && cleanup_repo ) || log_fatal
done <<< "$candidate_repo_dirs"
}
transfer_repo() {
local repo_name
local matches
local matches_count
local entry
local branch
local base
local namespace
local initialized
local integration_branch
local team_branch
repo_name="$(basename "$PWD")"
log_title "[transfer] ($repo_name)"
matches=( $(get_matching_branches_array "$repo_name") )
matches_count=${#matches[@]}
if (( matches_count == 0 )); then
log; log_info "no matching branches found, skipping transfer." 1
return 0
fi
log_subtitle "saving $matches_count branch entries to the local git config..." 1
for entry in "${matches[@]}"; do
branch="${entry%%$CONFIG_ENTRY_DELIMITER*}"
base="${entry##*$CONFIG_ENTRY_DELIMITER}"
namespace="branch:$branch.base"
git_config_set "$namespace" "$base"
status_log_fatal $? "saved ok [$namespace => $base]" "save failed [$namespace => $base]" 2
done
log_subtitle "saving initialized flag to the local git config..." 1
namespace="initialized"
initialized="$(git_config_global_get "$namespace")"
git_config_set "$namespace" "$initialized"
status_log_fatal $? "saved ok [$namespace => $initialized]" "save failed [$namespace => $initialized]" 2
log_subtitle "saving long-lived branches to the local git config..." 1
namespace="branch.integration"
integration_branch="$(git_config_global_get "$namespace")"
git_config_set "$namespace" "$integration_branch"
status_log_fatal $? "saved ok [$namespace => $integration_branch]" "save failed [$namespace => $integration_branch]" 2
namespace="branch.team"
team_branch="$(git_config_global_get "$namespace")"
git_config_set "$namespace" "$team_branch"
status_log_fatal $? "saved ok [$namespace => $team_branch]" "save failed [$namespace => $team_branch]" 2
}
cleanup_repo() {
local repo_name
local matches
local matches_count
local entry
local branch
local namespace
repo_name="$(basename "$PWD")"
log_title "[cleanup] ($repo_name)"
matches=( $(get_matching_branches_array "$repo_name") )
matches_count=${#matches[@]}
if (( matches_count == 0 )); then
log; log_info "no matching branches found, skipping cleanup." 1
return 0
fi
log_subtitle "removing $matches_count branch entries from the global git config..." 1
for entry in "${matches[@]}"; do
branch="${entry%%$CONFIG_ENTRY_DELIMITER*}"
namespace="branch:$branch"
git_v_config_global_remove_section "$namespace"
done
}
cleanup_long_lived_branch_entries() {
# skip this operation in partial migration
if is_git_repo; then
return 0
fi
log_title "[cleanup long-lived]"
log_subtitle "removing long-lived branches section from the global git config..." 1
git_v_config_global_remove_section "branch"
}
cleanup_orphan_branch_entries() {
local sections
local namespace
# skip this operation in partial migration
if is_git_repo; then
return 0
fi
log_title "[cleanup orphans]"
sections=( $(git_config_global_get_all_sections_matching 'branch:.*') )
log_subtitle "removing ${#sections[@]} orphan branch entries from the global git config..." 1
for namespace in "${sections[@]}"; do
# strip prefix from the beginning (it'll be re-added later)
namespace="${namespace#$GIT_CONFIG_SECTION.}"
# strip prop key from the end, we only want the subsection name
namespace="${namespace%.base}"
git_v_config_global_remove_section "$namespace"
done
}
cleanup_temp_dir() {
local output
local status
log_title "[cleanup temp]"
remove_dir "$TEMP_DIR"
}
purge_artifacts() {
case "$PURGE" in
backup)
purge_backup
;;
dryrun)
purge_dryrun
;;
all)
purge_backup
purge_dryrun
;;
esac
}
purge_backup() {
log_title "[purge backup]"
# skip this operation in partial migration
if is_git_repo; then
log; log_info "cannot operate on the repo-level, change to the workspace dir and try again." 1
return 0
fi
purge_config_artifacts "$BACKUP_EXT"
}
purge_dryrun() {
log_title "[purge dry-run]"
# skip this operation in partial migration
if is_git_repo; then
log; log_info "cannot operate on the repo-level, change to the workspace dir and try again." 1
return 0
fi
purge_config_artifacts "gitflux.dryrun"
}
purge_config_artifacts() {
local path_suffix="$1"
# global
remove_files <<< ~/.gitconfig.*"$path_suffix"
# local
remove_files <<< ./*/.git/config.*"$path_suffix"
}
restore_backup() {
local last_backup_suffix
local dryrun_suffix
local config_paths
log_title "[rollback]"
# skip this operation in partial migration
if is_git_repo; then
log; log_info "cannot operate on the repo-level, change to the workspace dir and try again." 1
return 0
fi
last_backup_suffix="$(get_last_backup_suffix)"
if (( $? != 0 )); then
log; log_info "no backups found, aborting." 1
return 0
fi
config_paths=$(list_config_paths)
if is_dry_run; then
dryrun_suffix=".$DRY_RUN_EXT"
fi
move_path "$last_backup_suffix" "$dryrun_suffix" true <<< "$config_paths"
}
get_last_backup_suffix() {
local last_global_backup
local last_backup_suffix
last_global_backup="$(get_last_file <<< ~/.gitconfig.*."$BACKUP_EXT")"
if [[ ! -e $last_global_backup ]]; then
return 1
fi
# strip path and file prefix
last_backup_suffix="${last_global_backup#*.gitconfig}"
printf "%s" "$last_backup_suffix"
return 0
}
# ----- trap handlers -----
on_exit() {
cleanup_temp_dir
log_title "[exit]"
log; log_info "backups were created for all config files, you can find them near the original files, with the '.$BACKUP_SUFFIX' extension." 1
log "to get rid of all those files, run again with PURGE=backup in the environment." 1
log; log_info "to restore all configs to the latest backup, run again with ROLLBACK=true in the environment." 1
if is_dry_run; then
log; log_info "this was a dry-run, check the *.$DRY_RUN_EXT files to see the changes." 1
log "to get rid of all those files, run again with PURGE=dryrun in the environment." 1
fi
log
}
# ----- cacheable operations -----
list_repositories() {
local status=0
local result
# search in cache
if [[ -e $TEMP_DIR/repo-dirs ]]; then
result="$(cat "$TEMP_DIR/repo-dirs")"
else
if is_git_repo; then
result="."
else
result="$(search_repo_dirs)"
status=$?
fi
# cache the results
print_lines <<< "$result" > "$TEMP_DIR/repo-dirs"
fi
printf "%s\n" "$result"
return $status
}
list_candidate_repositories() {
local status=0
local result
# search in cache
if [[ -e $TEMP_DIR/candidate-repo-dirs ]]; then
result="$(cat "$TEMP_DIR/candidate-repo-dirs")"
else
if is_git_repo; then
result="."
else
result="$(filter_candidate_repositories)"
status=$?
fi
# cache the results
print_lines <<< "$result" > "$TEMP_DIR/candidate-repo-dirs"
fi
printf "%s\n" "$result"
return $status
}
get_matching_branches_array() {
local repo="$1"
local status=0
local result
local branches
# search in cache
if [[ -e $TEMP_DIR/config-matches/$repo ]]; then
result=( $(cat "$TEMP_DIR/config-matches/$repo") )
else
branches=( $(git_local_branches) )
result=( $(filter_matching_branches "${branches[@]}") )
if (( ${#result[@]} > 0 )); then
# cache the results
ensure_dir "$TEMP_DIR/config-matches"
print_lines <<< "${result[@]}" > "$TEMP_DIR/config-matches/$repo"
fi
fi
printf "%s\n" "${result[@]}"
return $status
}
# ----- utilities -----
get_last_file() {
local files_glob
files_glob=$(cat -)
# shellcheck disable=SC2086
printf "%s\n" ${files_glob} | sort | tail -n 1
}
remove_dir() {
local dir="$1"
local output
local status
log_subtitle "deleting dir [$dir]" 1
output="$(rm -rv "${dir:?}" 2>&1)"
status=$?
output="$(print_lines 3 <<< "$output")"
status_log_notify $status "deleted ok:
$output" "delete failed:
$output" 2
}
remove_files() {
local files_glob
local output
local status
files_glob=$(cat -)
log_subtitle "deleting files [$files_glob]..." 1
# shellcheck disable=SC2086
output="$(rm -v ${files_glob:?} 2>&1)"
status=$?
output="${output#rm: }"
output="$(print_lines 3 <<< "$output")"
status_log_notify $status "deleted ok:
$output" "delete failed:
$output" 2
}
count_lines() {
local list
list="$(cat -)"
printf "%g" "$(printf "%s\n" "$list" | strip_blank_lines | wc -l)"
}
strip_blank_lines() {
egrep -v '^[[:blank:]]*$'
}
ensure_dir() {
local dir="$1"
if [[ ! -d $dir ]]; then
mkdir "$dir"
fi
}
copy_path() {
local src_suffix="$1"
local dest_suffix="$2"
local src
local dest
local paths
local output
paths="$(cat -)"
log_subtitle "copying $(count_lines <<< "$paths") files..." 1
while read -r path; do
src="$path$src_suffix"
dest="$path$dest_suffix"
output="$(cp -v "$src" "$dest" 2>&1)"
status_log_fatal $? "copied ok [$output]" "copy failed [$output]" 2
done <<< "$paths"
}
move_path() {
local src_suffix="$1"
local dest_suffix="$2"
local overwrite="$3"
local paths
local src
local dest
local output
paths="$(cat -)"
log_subtitle "moving files..." 1
while read -r path; do
src="$path$src_suffix"
dest="$path$dest_suffix"
# skip non-existing files; check dest too, if this is an overwrite operation
if [[ ! -e $src ]] || [[ $overwrite = true && ! -e $dest ]]; then
continue
fi
output="$(mv -v "$src" "$dest" 2>&1)"
status_log_notify $? "moved ok [$output]" "move failed [$output]" 2
done <<< "$paths"
}
search_repo_dirs() {
local dir
for dir in */; do
( cd "$dir" && is_git_repo ) 2>/dev/null
if (( $? == 0 )); then
printf "%s\n" "${dir%/}"
fi
done
}
filter_candidate_repositories() {
local repositories
repositories="$(list_repositories)"
while read -r repo; do
( cd "$repo" && is_candidate_repo "$repo" ) 2>/dev/null
if (( $? == 0 )); then
printf "%s\n" "${repo%/}"
fi
done <<< "$repositories"
}
is_candidate_repo() {
local repo="$1"
local matches
matches=( $(get_matching_branches_array "$repo") )
if (( ${#matches[@]} == 0 )); then
return 1
fi
return 0
}
# todo - cache repo branches?
filter_matching_branches() {
local branch
local base
for branch in "$@"; do
base="$(git_config_global_get "branch:$branch.base")"
if [[ $base ]]; then
printf "%s\n" "$branch$CONFIG_ENTRY_DELIMITER$base"
fi
done
}
git_local_branches() {
# no need for the master branch in our situation
git branch --no-color | sed -e 's,^[* ] ,,' | grep -v --color=never 'master'
}
list_config_paths() {
local repo_paths
if is_git_repo; then
# add the global config
printf "%s\n" ~/.gitconfig
# add local config
printf "%s\n" ./.git/config
else
# if repo paths are piped to the function, use them to filter the list
repo_paths="$(cat -)"
# attempt to discover available repositories as fallback
if [[ -z $repo_paths ]]; then
repo_paths="$(list_repositories)"
fi
# check again if we found anything
if [[ $repo_paths ]]; then
# add the global config
printf "%s\n" ~/.gitconfig
# add local repo configs
while read -r dir; do
printf "%s\n" "$dir"/.git/config
done <<< "$repo_paths"
fi
fi
}
is_git_repo() {
[[ -d .git ]] || git rev-parse --git-dir >/dev/null 2>&1
}
create_temp_dir() {
mktemp -d 2>/dev/null || mktemp -d -t 'git-flux-config-migration'
}
is_dry_run() {
[[ $DRY_RUN = true ]]
}
# ----- git-config utilities -----
git_config_local_context() {
local repo_path="$1"
local config_file
if is_dry_run; then
config_file="$repo_path"/.git/config."$DRY_RUN_EXT"
printf "%s" "--file $config_file"
fi
}
git_config_global_context() {
local config_file
if is_dry_run; then
config_file=~/.gitconfig."$DRY_RUN_EXT"
printf "%s" "--file $config_file"
else
printf "%s" "--global"
fi
}
git_config_set() {
local namespace="$1"
local value="$2"
local context
context=$(git_config_local_context "$PWD")
# shellcheck disable=SC2086
git config ${context} "$GIT_CONFIG_SECTION.$namespace" "$value"
}
git_config_global_get() {
local namespace="$1"
local context
context=$(git_config_global_context)
# ignore failures by discarding stderr - this function's output is used as an input
# shellcheck disable=SC2086
git config ${context} --get "$GIT_CONFIG_SECTION.$namespace" 2>/dev/null
}
git_config_global_get_all_sections_matching() {
local namespace="$1"
local context
context=$(git_config_global_context)
# ignore failures by discarding stderr - this function's output is used as an input list
# shellcheck disable=SC2086
git config ${context} --name-only --get-regexp "$GIT_CONFIG_SECTION.$namespace" 2>/dev/null
}
git_config_global_remove_section() {
local namespace="$1"
local context
context=$(git_config_global_context)
# shellcheck disable=SC2086
git config ${context} --remove-section "$GIT_CONFIG_SECTION.$namespace"
}
git_v_config_global_remove_section() {
local namespace="$1"
local output
local status
output="$(git_config_global_remove_section "$namespace" 2>&1)"
status=$?
output="${output#fatal: }"
status_log_notify $status "removed ok [$namespace]" "not removed [$namespace] ($output)" 2
}
# ----- logging -----
print() {
local margin="$1"
local message="$2"
local indent_level=$3
local indent
local n
if [[ $indent_level =~ [12345] ]]; then
for (( n=0; n<indent_level; n++ )); do
indent=" $indent"
done
fi
printf "%s\n" "$indent$margin$message"
}
print_lines() {
local indent_level=$1
local lines
lines="$(cat -)"
while read -r line; do
# shellcheck disable=SC2086
print "" "$line" $indent_level
done <<< "$lines"
}
log() {
# shellcheck disable=SC2086
print " " "$1" $2
}
log_ok() {
# shellcheck disable=SC2086
print " ✔ " "$1" $2
}
log_info() {
# shellcheck disable=SC2086
print " ℹ " "$1" $2
}
log_err() {
# shellcheck disable=SC2086
print " ✘ " "$1" $2
}
log_fatal() {
local default_msg="something terrible happened! aborting. run again with ROLLBACK=true in the environment to revert changes."
local msg="${1:-$default_msg}"; shift
log; log_err "$msg" "$@"; log
exit 1
}
log_title() {
log; log; log "$@"
}
log_subtitle() {
log; log "$@"
}
status_log_fatal() {
local status=$1
local ok_msg="$2"
local err_msg="$3"
local indent_level=$4
if (( status == 0 )); then
# shellcheck disable=SC2086
log_ok "$ok_msg" $indent_level
elif [[ $err_msg ]]; then
# shellcheck disable=SC2086
log_fatal "$err_msg" $indent_level
fi
}
status_log_notify() {
local status=$1
local ok_msg="$2"
local info_msg="$3"
local indent_level=$4
if (( status == 0 )); then
# shellcheck disable=SC2086
log_ok "$ok_msg" $indent_level
elif [[ $info_msg ]]; then
# shellcheck disable=SC2086
log_info "$info_msg" $indent_level
fi
}
main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment