In your package.json scripts... add these without newlines/escapes
"submodulePostCheckoutCorrectDetachedHeads": "
git submodule sync --recursive &&
git submodule update --checkout &&
git submodule foreach \
'latest_commits_remote_branch=$(\
git branch -r --contains `git log -1 --pretty=format:\"%h\"`
) node $toplevel/dev-scripts/checkout_right_branch $sm_path'", // end submodule post-checkout
"submodulePostMergeUpdateSubmoduleMerge": "
echo \"(this script used to also be run postinstall to ease setup) Running submoduleUpdateMergeIntoBranchesInGitmodules npm script\" &&
git submodule update --init --recursive --merge --no-fetch --remote &&
git submodule sync --recursive && git submodule foreach \
'git checkout `git config -f $toplevel/.gitmodules submodule.$sm_path.branch`' &&
git submodule update --init --recursive --merge &&
echo \"submoduleUpdateMergeIntoBranchesInGitmodules npm script completed successfully\"",
In husky post-checkout+post-merge, run these scripts (npm run submoduleFooBar). You can also put the submodule post merge in the postinstall npm script (scripts.postinstall - runs after npm install)
Git has a bug (as of 2.24) where push.recurseSubmodules = on-demand doens't work
You can have a pre-push hook where it works around this issue, but in vscode, pushes will always APPEAR to fail, but all submodules will actually get pushed. Pushing 2nd time (after successful push) will not "appear" to fail again, it will appear successful.
"pre-push-hook-that-recursively-pushed-submodules": "echo \"husky pre-push hook (has git hook .sample)\" && CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD) && git push -u origin $CURRENT_BRANCH --recurse-submodules=on-demand --no-verify",
You probably want a pre-commit hook that warns you once about commiting to your submodule master branch, if you are trying to enforce a pr workflow
Also on preinstall/after the repository clones, inside the project folder, run this on mac/linux:
git config --local include.path ../dev-scripts/.gitconfig
On windows run:
git config --local include.path $PWD/dev-scripts/.gitconfig
With this .gitconfig inside your project
# you need to run
# git config --local include.path ../.gitconfig
# for git to incorporate this config
# it's run automatically on npm install via the package.json postinstall script
# These are more unique and probably should not be set in your global config
[status]
submodulesummary = 1
[submodule]
recurse = true
[uploadpack]
allowReachableSHA1InWant = true
[blame]
ignoreRevsFile = .git-blame-ignore-revs
[core]
ignorecase = false
autocrlf = false # https://git-scm.com/docs/gitattributes#_end_of_line_conversion
[push]
recurseSubmodules = check
# recurseSubmodules = on-demand # this setting makes git push fail in superproject like so:
# git push git:dev*
# husky > pre-push (node v12.10.0)
# husky pre-push hook (has git hook .sample)
# fatal: src refspec 'refs/heads/dev' must name a ref
# fatal: process for submodule 'src/app/q360-comps/animated-meter' failed
# If enabling git "large file system", .gitattributes file should also have binary file modified like so:
# Example: *.ai filter=lfs diff=lfs merge=lfs binary
# See https://rehansaeed.com/gitattributes-best-practices/
# [filter "lfs"]
# process = git-lfs filter-process
# required = true
# clean = git-lfs clean -- %f
# smudge = git-lfs smudge -- %f
I recommend having only one submodule, ./packages, which holds all shared code.
Instead of forking I recommend duplicating folders inside this shared git repo.
It'd be great to have a npm track the git hash of a sibling folder (npm install ../shared-code).
It'd be also great to extend git to be able to allow sibling submodules.
If you are using submodules you are perhaps sharing code between multiple projects.
If you want to work on that submodule and have changes apply to both projects at the same time, you need git to follow symlinks.
Review code before pushing:
git diff origin/dev --submodule=diff -- ':(exclude)build' ':!package-lock.json' ':!*.svg' ':!cached-responses'
Reviewing code before pulling...
git fetch (or git fetch origin <branch>)
git diff FETCH_HEAD --submodule=diff -- ':(exclude)build' ':!package-lock.json' ':!*.svg' ':!cached-responses'
Optionally review all commits with commit messages+diffs
git log HEAD..FETCH_HEAD -p --submodule=diff -- ':(exclude)build' ':!package-lock.json' ':!*.svg' ':!cached-responses'
checkout_commits_branch.js
/* globals process */
const branch = process.env.latest_commits_branch.trim()
const [remote, branchName] = branch.split('/')
console.log('remote', remote, 'branchName', branchName, 'cleanExec', cleanExec)
console.log('pwd arg', '0', process.argv0, '1', process.argv1, '2', process.argv2, '3', process.argv3)
const cleanExec = require('./cleanExec')
cleanExec('pwd', wd => {
console.log('pwd inside node', wd)
})
// maybe async version slows your computer down less.
// cleanExec('git rev-parse --abbrev-ref HEAD', currentBranch => {
// if (currentBranch.trim() === 'master') {
// console.error('you cant be on master branch')
// return process.exit(1)
// }
// cleanExec('git status --porcelain', stdout => {
// stdout = stdout.replace(' M cached-responses', '').trim()
// if (stdout === '') {
// console.log('git status is clean.')
// } else {
// cleanExec('git status', gitStatus => {
// console.log(gitStatus)
// console.error('\n **COMMIT THESE CHANGES BEFORE CREATING BUILD**')
// process.exit(1)
// })
// }
// })
// })
checkout_right_branch.js
// used only inside npm run
// was originally intended to be run like so, making way for more features:
// git submodule foreach 'cd $toplevel && configured_branch=$(git config submodule.$sm_path.branch) && latest_commits_remote_branch=$(git branch -r --contains `git log -1 --pretty=format:\"%h\"`) && node $toplevel/dev-scripts/checkout-right-branch $sm_path'
// but configured_branch stuff was removed due to an error (which I didn't have time to investigate, since it's not necessary)
// so it's like this:
// git submodule foreach 'echo $(pwd) && echo $toplevel && latest_commits_remote_branch=$(git branch -r --contains `git log -1 --pretty=format:"%h"`) node $toplevel/dev-scripts/checkout_right_branch $sm_path'
// submodule_checkout
// full thing:
/*
git config --local include.path ../dev-scripts/.gitconfig && git submodule sync --recursive && git submodule update --checkout && git submodule foreach 'echo $(pwd) && echo $toplevel && latest_commits_remote_branch=$(git branch -r --contains `git log -1 --pretty=format:"%h"`) node $toplevel/dev-scripts/checkout_right_branch $sm_path' node $toplevel/dev-scripts/checkout_right_branch $sm_path'
*/
/* globals process */
console.log('latest_commits_remote_branch:start:' + process.env.latest_commits_remote_branch + ':END')
const branchContainsCommit = process.env.latest_commits_remote_branch.trim()
let branch
if (branchContainsCommit.includes('\n')) {
branch = branchContainsCommit.split('\n').pop()
} else {
branch = branchContainsCommit
}
if (branch.split('/').length !== 2) {
throw new Error(
'dev-scripts/checkout_right_branch.js expected branch to be like ` origin/feature-foo` but it was:',
branch
)
}
branch = branch.trim()
const [remote, branchName] = branch.split('/')
// console.log('remote', remote, 'branchName', branchName)
// console.log('pwd arg', '0', process.argv0, '1', process.argv1, '2', process.argv2, '3', process.argv3)
// eslint-disable-next-line security/detect-child-process
const { exec } = require('child_process')
// maybe async version slows your computer down less.
function cleanExecNoThrow(cmd, callback = () => {}) {
console.error(`executing: ${cmd}`)
exec(cmd, function execCompleted(err, stdout, stderr) {
// if (err !== null) {
// console.error(`nodejs err:`, err)
// }
if (stderr !== '') {
console.error(`stderr:`, stderr)
if (callback) callback(stderr, 'is_stderr')
}
if (callback) callback(stdout)
})
}
// usage: cleanExec('git status --porcelain', stdout => {
cleanExecNoThrow(
`cd $toplevel && cd $sm_path && git checkout -b ${branchName} --track ${branch}`,
(checkoutAttempt, is_stderr) => {
if (!is_stderr) console.log(checkoutAttempt)
if (checkoutAttempt.includes("fatal: A branch named '" + branchName + "' already exists.")) {
cleanExecNoThrow(
`cd $toplevel && cd $sm_path && git checkout ${branchName}`,
(checkoutAttempt2, is_stderr2) => {
if (!is_stderr2) console.log(checkoutAttempt2)
}
)
}
}
)
cleanExec.js
/* globals module */
// eslint-disable-next-line security/detect-child-process
const { exec } = require('child_process')
// maybe async version slows your computer down less.
module.exports = function cleanExec(cmd, callback) {
exec(cmd, function execCompleted(err, stdout, stderr) {
if (err !== null || stderr !== '') {
if (err !== null) {
console.error(`nodejs err exec'uting ${cmd}:`, err)
}
if (stderr !== '') {
console.error(`error exec'uting ${cmd}:`, stderr)
}
throw new Error('see logs above.')
}
if (callback) callback(stdout)
})
}
// usage: cleanExec('git status --porcelain', stdout => {
setCorrectSubmoduleBranches.js
/* globals process */
const cleanExec = require('./cleanExec')
cleanExec('pwd', wd => {
console.log('pwd inside node', wd)
const branch = process.env.latest_commits_branch.trim()
const [remote, branchName] = branch.split('/')
console.log('remote', remote, 'branchName', branchName, 'cleanExec', cleanExec)
console.log('arg0', process.argv0, '1', process.argv1, '2', process.argv2, '3', process.argv3)
})
// git submodule sync --recursive
// git submodule update --checkout
// git submodule foreach 'git checkout `git config -f $toplevel/.gitmodules submodule.$sm_path.branch`'
// git config --local include.path ../dev-scripts/.gitconfig
// latest_commits_branch=$(git branch -r --contains `git log -1 --pretty=format:"%h"`)
// latest_commits_branch=$(git branch -r --contains `git log -1 --pretty=format:"%h"`) && echo "current dir:${pwd}" && pwd && echo "sha? $1" &&
// latest_commits_branch=$(git branch -r --contains `git log -1 --pretty=format:"%h"`)
// echo "current dir:${pwd}"
// echo "sha? $1"
I tried getting a custom update script to work. I learned you are best of doing this via local .gitconfig file, which is why I have one of those. However, probably best to include .gitmodules file along with .gitconfig file like so
git config --local include.path ../.gitmodules
Then you are REALLY telling git to listen to the damn .gitmodules file, instead of relying on it to copy submodule configs into .git/config
By default custom commands write to the .git/config file as update = none which is beyond useless.
The community does absolutely need a "smart submodule checkout" which retains the branch a submodule is on.
However, submodules are currently greatly limited by not recording which branch a submodule should be on.
Here's a new idea...
You include the file just like the .gitmodules file and .gitconfig file above.
But all it contains are the branch names you want.
It's .gitignore'd
Because maybe you don't want to clutter history with merge commits. But come to think of it, probably better to have merge commits, and maybe also a submodule merge commit
Say you branch submoduleA, feature-foo
you also branch super/parent/container with same branch name, feature-foo
Really, you want a slick oneliner to do these two things and a third thing... change your submodules branch setting in .gitmodules or .gitmodulebranches
this way if a co-worker checks out the feature-foo branch, it is explicit which branch to be on
It may be better solved with a branch selection heuristic.
If a commit hash exists on the branch specified in .gitmodules, use that branch, which defaults to master
Although I don't think detached head is that hard to solve. If you pick the wrong branch, the your co-worker will see that (as long as you update the super/parent/container pointer to your the "wrong" branch you are committing on)
I have a few more pages of not-well-put-together notes.
But really the best thing would be to have a normal monorepo, and not fork codebases, where you then want to share/sync code via submodules
Just, figure out how to do it in one repo.
If you can't, hit me up, I'd love to help you figure out proper a good sharing architecture. My email is my github username @ gmail.com