Skip to content

Instantly share code, notes, and snippets.

@pmrowla
Created June 30, 2021 02:03
Show Gist options
  • Save pmrowla/c1e86bc41acc05d06a3752d8e8700e4a to your computer and use it in GitHub Desktop.
Save pmrowla/c1e86bc41acc05d06a3752d8e8700e4a to your computer and use it in GitHub Desktop.
dvc chained/circular import example
#!/bin/bash
# set -e
# set -x
ROOT=`pwd`
REPO=$ROOT/repo
REMOTE=$ROOT/remote
REPOS=(a b c d)
rm -rf $REPO $REMOTE
for repo in ${REPOS[@]}
do
mkdir -p $REPO/$repo
mkdir -p $REMOTE/$repo
pushd $REPO/$repo
git init
dvc init
dvc remote add -d myremote $REMOTE/$repo
git add .
git commit -m 'init'
popd
done
pushd $REPO/a
echo -n foo > foo
dvc add foo
git add .
git commit -m 'add foo'
dvc push
popd
pushd $REPO/b
echo -n bar > bar
dvc add bar
git add .
git commit -m 'add bar'
dvc push
popd
pushd $REPO/c
mkdir -p dir/subdir
echo -n baz > dir/subdir/baz
dvc add dir/subdir
git add .
git commit -m 'add subdir/baz'
dvc import -o dir/foo $REPO/a foo
dvc import -o dir/bar $REPO/b bar
git add .
git commit -m 'intermediate import'
dvc push
popd
pushd $REPO/d
dvc import $REPO/c dir
git add .
git commit -m 'chained import'
dvc push
popd
echo ""
tree $REPO/a
echo ""
tree $REPO/b
echo ""
tree $REPO/c
cat $REPO/c/dir/*.dvc
echo ""
tree $REPO/d
cat $REPO/d/dir.dvc
pushd $REPO/d
rm -rf .dvc/cache
dvc pull -v
popd
pushd $REPO/a
dvc import $REPO/d dir
popd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment