Skip to content

Instantly share code, notes, and snippets.

@shcheklein
Created September 11, 2019 17:05
Show Gist options
  • Save shcheklein/e8aec7cf438c685b496ca2cffd280b58 to your computer and use it in GitHub Desktop.
Save shcheklein/e8aec7cf438c685b496ca2cffd280b58 to your computer and use it in GitHub Desktop.
2483 test
(.env) [ivan@ivan ~/Projects/test]$ ./test.sh
+ set -e
+ pip uninstall -y dvc
Uninstalling dvc-0.59.2:
Successfully uninstalled dvc-0.59.2
+ pip install dvc==0.59.2
Collecting dvc==0.59.2
Using cached https://files.pythonhosted.org/packages/78/fa/5d9fc54fbf99c75efd9cfbd81b479b2e3d0572b59694523d24c280cad018/dvc-0.59.2-py2.py3-none-any.whl
Requirement already satisfied: future>=0.16.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.17.1)
Requirement already satisfied: asciimatics>=1.10.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.11.0)
Requirement already satisfied: configparser>=3.5.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (4.0.1)
Requirement already satisfied: funcy>=1.12 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.13)
Requirement already satisfied: requests>=2.22.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (2.22.0)
Requirement already satisfied: configobj>=5.0.6 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (5.0.6)
Requirement already satisfied: humanize>=0.5.1 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.1)
Requirement already satisfied: shortuuid>=0.5.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.0)
Requirement already satisfied: inflect>=2.1.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (2.1.0)
Requirement already satisfied: distro>=1.3.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.0)
Requirement already satisfied: schema>=0.6.7 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.7.1)
Requirement already satisfied: jsonpath-ng>=1.4.3 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.3)
Requirement already satisfied: pyasn1>=0.4.1 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.4.7)
Requirement already satisfied: colorama>=0.3.9 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.4.1)
Requirement already satisfied: grandalf==0.6 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.6)
Requirement already satisfied: pathspec>=0.5.9 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.9)
Requirement already satisfied: treelib>=1.5.5 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.5.5)
Requirement already satisfied: setuptools>=34.0.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (41.2.0)
Requirement already satisfied: ply>=3.9 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (3.11)
Requirement already satisfied: ruamel.yaml>=0.16.1 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.16.5)
Requirement already satisfied: tqdm>=4.35.0 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (4.35.0)
Requirement already satisfied: nanotime>=0.5.2 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.2)
Requirement already satisfied: gitpython>=2.1.8 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (3.0.2)
Requirement already satisfied: networkx>=2.1 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (2.3)
Requirement already satisfied: appdirs>=1.4.3 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.3)
Requirement already satisfied: zc.lockfile>=1.2.1 in ./.env/lib/python3.7/site-packages (from dvc==0.59.2) (2.0)
Requirement already satisfied: Pillow>=2.7.0 in ./.env/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (6.1.0)
Requirement already satisfied: wcwidth in ./.env/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (0.1.7)
Requirement already satisfied: pyfiglet>=0.7.2 in ./.env/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (0.8.post1)
Requirement already satisfied: certifi>=2017.4.17 in ./.env/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (2019.6.16)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.env/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (1.25.3)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.env/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in ./.env/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (2.8)
Requirement already satisfied: six in ./.env/lib/python3.7/site-packages (from configobj>=5.0.6->dvc==0.59.2) (1.12.0)
Requirement already satisfied: contextlib2==0.5.5 in ./.env/lib/python3.7/site-packages (from schema>=0.6.7->dvc==0.59.2) (0.5.5)
Requirement already satisfied: decorator in ./.env/lib/python3.7/site-packages (from jsonpath-ng>=1.4.3->dvc==0.59.2) (4.4.0)
Requirement already satisfied: pyparsing in ./.env/lib/python3.7/site-packages (from grandalf==0.6->dvc==0.59.2) (2.4.2)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" in ./.env/lib/python3.7/site-packages (from ruamel.yaml>=0.16.1->dvc==0.59.2) (0.1.2)
Requirement already satisfied: gitdb2>=2.0.0 in ./.env/lib/python3.7/site-packages (from gitpython>=2.1.8->dvc==0.59.2) (2.0.5)
Requirement already satisfied: smmap2>=2.0.0 in ./.env/lib/python3.7/site-packages (from gitdb2>=2.0.0->gitpython>=2.1.8->dvc==0.59.2) (2.0.5)
Installing collected packages: dvc
Successfully installed dvc-0.59.2
+ rm -rf dvc_test
+ mkdir dvc_test
+ cd dvc_test
+ mkdir scripts
+ mkdir -p data/recommended/dataset1/dataset1_proc
+ echo bar
+ git init
Initialized empty Git repository in /Users/ivan/Projects/test/dvc_test/.git/
+ dvc init
Adding '.dvc/lock' to '.dvc/.gitignore'.
Adding '.dvc/config.local' to '.dvc/.gitignore'.
Adding '.dvc/updater' to '.dvc/.gitignore'.
Adding '.dvc/updater.lock' to '.dvc/.gitignore'.
Adding '.dvc/state-journal' to '.dvc/.gitignore'.
Adding '.dvc/state-wal' to '.dvc/.gitignore'.
Adding '.dvc/state' to '.dvc/.gitignore'.
Adding '.dvc/cache' to '.dvc/.gitignore'.
You can now commit the changes to git.
+---------------------------------------------------------------------+
| |
| DVC has enabled anonymous aggregate usage analytics. |
| Read the analytics documentation (and how to opt-out) here: |
| https://dvc.org/doc/user-guide/analytics |
| |
+---------------------------------------------------------------------+
What's next?
------------
- Check out the documentation: https://dvc.org/doc
- Get help and share ideas: https://dvc.org/chat
- Star us on GitHub: https://github.com/iterative/dvc
+ echo -e 'import sys\nwith open(sys.argv[1], '\''w'\'') as f: f.write(sys.argv[2])'
+ dvc run -w ./data/recommended/dataset1/dataset1_proc -f ./data/recommended/dataset1/dataset1_proc/v1.dvc -d ../../../../scripts/script.py -o v1 'mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data'
Running command:
mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data
Adding 'data/recommended/dataset1/dataset1_proc/v1' to 'data/recommended/dataset1/dataset1_proc/.gitignore'.
Saving 'data/recommended/dataset1/dataset1_proc/v1' to '.dvc/cache/18/8ed6cb603658d01ef7ba8fb7c434fe.dir'.
Saving information to 'data/recommended/dataset1/dataset1_proc/v1.dvc'.
To track the changes with git, run:
git add data/recommended/dataset1/dataset1_proc/.gitignore data/recommended/dataset1/dataset1_proc/v1.dvc
+ cat data/recommended/dataset1/dataset1_proc/v1.dvc
md5: 1f92bcfcf1ef2766da1ea3c373289dca
cmd: mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data
deps:
- md5: 791b9c74b1d9308a3226b93a36689dad
path: ../../../../scripts/script.py
outs:
- md5: 188ed6cb603658d01ef7ba8fb7c434fe.dir
path: v1
cache: true
metric: false
persist: false
+ dvc repro data/recommended/dataset1/dataset1_proc/v1.dvc
Stage 'data/recommended/dataset1/dataset1_proc/v1.dvc' didn't change.
Data and pipelines are up to date.
Copy link

ghost commented Sep 11, 2019

Mine:

❯ ./script.sh
+ set -e
+ pip uninstall -y dvc
Uninstalling dvc-0.59.2:
  Successfully uninstalled dvc-0.59.2
+ pip install dvc==0.59.2
Collecting dvc==0.59.2
  Using cached https://files.pythonhosted.org/packages/78/fa/5d9fc54fbf99c75efd9cfbd81b479b2e3d0572b59694523d24c280cad018/dvc-0.59.2-py2.py3-none-any.whl
Requirement already satisfied: jsonpath-ng>=1.4.3 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.3)
Requirement already satisfied: tqdm>=4.35.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (4.35.0)
Requirement already satisfied: future>=0.16.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.17.1)
Requirement already satisfied: ply>=3.9 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (3.11)
Requirement already satisfied: nanotime>=0.5.2 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.2)
Requirement already satisfied: ruamel.yaml>=0.16.1 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.16.5)
Requirement already satisfied: requests>=2.22.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (2.22.0)
Requirement already satisfied: asciimatics>=1.10.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.11.0)
Requirement already satisfied: setuptools>=34.0.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (40.8.0)
Requirement already satisfied: treelib>=1.5.5 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.5.5)
Requirement already satisfied: inflect>=2.1.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (2.1.0)
Requirement already satisfied: pyasn1>=0.4.1 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.4.7)
Requirement already satisfied: shortuuid>=0.5.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.0)
Requirement already satisfied: humanize>=0.5.1 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.1)
Requirement already satisfied: gitpython>=2.1.8 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (3.0.2)
Requirement already satisfied: configobj>=5.0.6 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (5.0.6)
Requirement already satisfied: networkx>=2.1 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (2.3)
Requirement already satisfied: configparser>=3.5.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (4.0.1)
Requirement already satisfied: grandalf==0.6 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.6)
Requirement already satisfied: zc.lockfile>=1.2.1 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (2.0)
Requirement already satisfied: appdirs>=1.4.3 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.3)
Requirement already satisfied: funcy>=1.12 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.13)
Requirement already satisfied: schema>=0.6.7 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.7.1)
Requirement already satisfied: pathspec>=0.5.9 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.5.9)
Requirement already satisfied: distro>=1.3.0 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (1.4.0)
Requirement already satisfied: colorama>=0.3.9 in ./.venv/lib/python3.7/site-packages (from dvc==0.59.2) (0.4.1)
Requirement already satisfied: six in ./.venv/lib/python3.7/site-packages (from jsonpath-ng>=1.4.3->dvc==0.59.2) (1.12.0)
Requirement already satisfied: decorator in ./.venv/lib/python3.7/site-packages (from jsonpath-ng>=1.4.3->dvc==0.59.2) (4.4.0)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" in ./.venv/lib/python3.7/site-packages (from ruamel.yaml>=0.16.1->dvc==0.59.2) (0.1.2)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (2019.6.16)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.venv/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (1.25.3)
Requirement already satisfied: idna<2.9,>=2.5 in ./.venv/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.venv/lib/python3.7/site-packages (from requests>=2.22.0->dvc==0.59.2) (3.0.4)
Requirement already satisfied: pyfiglet>=0.7.2 in ./.venv/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (0.8.post1)
Requirement already satisfied: wcwidth in ./.venv/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (0.1.7)
Requirement already satisfied: Pillow>=2.7.0 in ./.venv/lib/python3.7/site-packages (from asciimatics>=1.10.0->dvc==0.59.2) (6.1.0)
Requirement already satisfied: gitdb2>=2.0.0 in ./.venv/lib/python3.7/site-packages (from gitpython>=2.1.8->dvc==0.59.2) (2.0.5)
Requirement already satisfied: pyparsing in ./.venv/lib/python3.7/site-packages (from grandalf==0.6->dvc==0.59.2) (2.4.2)
Requirement already satisfied: contextlib2==0.5.5 in ./.venv/lib/python3.7/site-packages (from schema>=0.6.7->dvc==0.59.2) (0.5.5)
Requirement already satisfied: smmap2>=2.0.0 in ./.venv/lib/python3.7/site-packages (from gitdb2>=2.0.0->gitpython>=2.1.8->dvc==0.59.2) (2.0.5)
Installing collected packages: dvc
Successfully installed dvc-0.59.2
You are using pip version 19.0.3, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ rm -rf dvc_test
+ mkdir dvc_test
+ cd dvc_test
+ mkdir scripts
+ mkdir -p data/recommended/dataset1/dataset1_proc
+ echo bar
+ git init
Initialized empty Git repository in /home/mroutis/tmp/dvc_test/.git/
+ dvc init
Adding '.dvc/lock' to '.dvc/.gitignore'.
Adding '.dvc/config.local' to '.dvc/.gitignore'.
Adding '.dvc/updater' to '.dvc/.gitignore'.
Adding '.dvc/updater.lock' to '.dvc/.gitignore'.
Adding '.dvc/state-journal' to '.dvc/.gitignore'.
Adding '.dvc/state-wal' to '.dvc/.gitignore'.
Adding '.dvc/state' to '.dvc/.gitignore'.
Adding '.dvc/cache' to '.dvc/.gitignore'.

You can now commit the changes to git.

What's next?
------------
- Check out the documentation: https://dvc.org/doc
- Get help and share ideas: https://dvc.org/chat
- Star us on GitHub: https://github.com/iterative/dvc
+ echo -e 'import sys\nwith open(sys.argv[1], '\''w'\'') as f: f.write(sys.argv[2])'
+ dvc run -w ./data/recommended/dataset1/dataset1_proc -f ./data/recommended/dataset1/dataset1_proc/v1.dvc -d ../../../../scripts/script.py -o v1 'mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data'
Running command:
	mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data
Adding 'data/recommended/dataset1/dataset1_proc/v1' to 'data/recommended/dataset1/dataset1_proc/.gitignore'.
Saving 'data/recommended/dataset1/dataset1_proc/v1' to '.dvc/cache/18/8ed6cb603658d01ef7ba8fb7c434fe.dir'.
Saving information to 'data/recommended/dataset1/dataset1_proc/v1.dvc'.

To track the changes with git, run:

	git add data/recommended/dataset1/dataset1_proc/v1.dvc data/recommended/dataset1/dataset1_proc/.gitignore
+ cat data/recommended/dataset1/dataset1_proc/v1.dvc
md5: 1f92bcfcf1ef2766da1ea3c373289dca
cmd: mkdir v1 && python ../../../../scripts/script.py v1/v1.txt proc_data
deps:
- md5: 791b9c74b1d9308a3226b93a36689dad
  path: ../../../../scripts/script.py
outs:
- md5: 188ed6cb603658d01ef7ba8fb7c434fe.dir
  path: v1
  cache: true
  metric: false
  persist: false
+ dvc repro data/recommended/dataset1/dataset1_proc/v1.dvc
Stage 'data/recommended/dataset1/dataset1_proc/v1.dvc' didn't change.
Data and pipelines are up to date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment