de-submodule-ification

#!/bin/sh

set -o errexit
set -o nounset
set -o xtrace

# submodules without submodules of their own are easier
for sm in Conditions Ecal Framework Hcal MagFieldMap TrigScint cmake
do
  sm_main="$(git -C ${sm} remote show origin | sed -n '/HEAD branch/s/.*: //p')"
  git remote add ${sm}_origin git@github.com:LDMX-Software/${sm}.git
  git fetch ${sm}_origin ${sm_main}
  git merge -s ours --no-commit --allow-unrelated-histories ${sm}_origin/${sm_main}
  git rm --cached ${sm}
  sed -i "/${sm}/d" .gitmodules
  rm -rf ${sm}/.git
  git add ${sm} .gitmodules
  git commit -m "transition ${sm} from submodule to subdirectory"
  git remote rm ${sm}_origin
done

# this method of copying the commit reference of the submodules of the submodules
# disregards the commit history of that submodule :(
for sm in SimCore Tracking Trigger
do
  git -C ${sm} submodule status > ${sm}.submods.list
  git -C ${sm} submodule --quiet foreach git remote get-url origin > ${sm}.submods.remote
  sm_main="$(git -C ${sm} remote show origin | sed -n '/HEAD branch/s/.*: //p')"
  git remote add ${sm}_origin git@github.com:LDMX-Software/${sm}.git
  git fetch ${sm}_origin ${sm_main}
  git merge -s ours --no-commit --allow-unrelated-histories ${sm}_origin/${sm_main}
  git rm --cached ${sm}
  sed -i "/${sm}/d" .gitmodules
  rm -rf ${sm}/.git .git/modules/${sm}
  paste ${sm}.submods.list ${sm}.submods.remote | while read commit relpath describe url
    do
      rm -rf "${sm}/${relpath}"
      git submodule add "${url}" "${sm}/${relpath}"
      git -C "${sm}/${relpath}" checkout "${commit}"
      git add "${sm}/${relpath}"
    done
  git add ${sm} .gitmodules
  git commit -m "transition ${sm} from submodule to subdirectory"
  git remote rm ${sm}_origin
  rm ${sm}.submods.list ${sm}.submods.remote
done

notes.md

Day-Of Procedure

• New release of submodules if HEAD of default branch differs
• Merge new submodule versions
• Make v3.4.0 release of ldmx-sw
• Merge new gold histograms
• New clone of ldmx-sw
• Apply de-submodule-ification script
• Puse updated trunk
• Run PR Validation tests with simple cleanup PR
• (if pass) Make v4.0.0 release of ldmx-sw

After Transition

• Update Ecal:trig-prim-study to be Ecal/trig-prim-study and re-open draft PR
• Update 1232-use-denv-in-ldmx-env
• Local test of wketchum/** in cmake and SimCore

de-submodule-ification

Converting submodules into regular directories while trying to maintain some semblance of history. I did some research on different ways to do this (below) and I've chosen the merge option which is implemented in the de-submodule-ification script. I had to modify it a bit since we have submodules within our submodules. The reset script is one I used to reset the ldmx-sw working tree while developing the de-submodule-ification script after it failed to successfully run. Resetting after a successful run is easier and is just rm -rf ldmx-sw && git clone --recursive git@github.com:LDMX-Software/ldmx-sw.git

Fully Merge Files and History

• oliver-la

I don't like this because it is a destructive action. In an attempt to keep the repository with a single history branch, it rewrites all commits related to the submodules and the files within them in the main repo. This would require a force push to ldmx-sw:trunk which I'm not willing to do or the creation of a new repository which would probably lead to more confusion.

rewrite submodule history

• git-submodule-rewrite
• blog post above script built off of

These only use filter-branch type commands on local copies of the submodules so that the commits from those submodules can be re-written with the updated paths.

Have not gotten the script to work

merge

• stackoverflow answer explaining this

git remote add submod_origin git@github.com:LDMX-Software/submod.git
git fetch submod_origin
git merge -s ours --no-commit --allow-unrelated-histories submod_origin/trunk
git rm --cached submod
vim .gitmodules # remove submod section
git add .gitmodules
rm -rf submod/.git
git add submod
git commmit -m "transitioned 'submod' from submodule into subdirectory"
git remote rm submod_origin

no history

Basically same as merge method, but avoiding the pollution of the history by just giving up on the history. Deleting the submodule and adding all of its files as new files in one commit.

git rm --cached submod
vim .gitmodules # remove submod section
git add .gitmodules
rm -rf submod/.git
git add submod
git commmit -m "transitioned 'submod' from submodule into subdirectory"

reset

#!/bin/sh

for sm in Conditions Ecal Framework Hcal MagFieldMap TrigScint cmake SimCore Tracking Trigger
do
  git remote rm ${sm}_origin
done
git merge --abort
git reset --hard origin/trunk
git submodule update --init --recursive