doit dry-run design

doit-dryrun-design.md

doit DRY-RUN

note use-case result columns:

• expected: expected by schettion72 :)
• doit: as given by Oliver branch

single task with file_dep and target

dodo.py

def task_single():
    return {
        'file_dep': ["existing.txt"],
        'actions': ["cat existing.txt > output1.txt"],
        'targets': ["output1.txt"],
    }

Makefile

output1.txt: existing.txt
	cat existing.txt > output1.txt

$ doit --dry-run output1.txt
$ make -n output1.txt

use case	expected	make	doit
file dep doesnt exist	ERROR[1]	ERROR[2]	run
task up-to-date	up-to-date	up-to-date	up-to-date
task not up-to-date	run	run	run

[1] Exception: Dependent file 'existing.txt' does not exist.

[2] make: *** No rule to make target existing.txt', needed by output1'. Stop.

task1.target == task2.file_dep | task1.update == True

dodo.py

def task_chain():
    yield {'name': 't1',
           'targets'   : ["tmp1"],
           'actions'   : ["echo hi1 > tmp1"],
           'uptodate': [True],
       }
    yield {'name': 't2',
           'file_dep'  : ["tmp1"],
           'targets'   : ["tmp2"],
           'actions'   : ["echo hi2 > tmp2"],
       }

Makefile

tmp1:
	echo hi1 > tmp1

tmp2: tmp1
	echo hi2 > tmp2

$ doit --dry-run chain
$ make -n tmp2

use case	expected	make	doit
tmp1 & tmp2 dont exist	run	run	run
tmp1 & tmp2 exist	up-to-date	up-to-date	up-to-date
tmp2 doesnt exist	run t2	run t2	run t2

task1.target == task2.file_dep | make task1 is phony

dodo.py

def task_chain_phony():
    yield {'name': 't1',
           'targets'   : ["tmp1_phony"],
           'actions'   : ["echo hi1 > tmp1_phony"],
       }
    yield {'name': 't2',
           'file_dep'  : ["tmp1_phony"],
           'targets'   : ["tmp2_phony"],
           'actions'   : ["echo hi2 > tmp2_phony"],
       }

Makefile

.PHONY: tmp1_phony
tmp1_phony:
	echo hi1 > tmp1_phony

tmp2_phony: tmp1_phony
	echo hi2 > tmp2_phony

$ doit --dry-run chain_phony
$ make -n tmp2_phony

use case	expected	make	doit
tmp1 & tmp2 dont exist	run	run	run
tmp1 & tmp2 exist	run	run	run

• note that without dry-run doit would re-execute tmp2 if tmp1 changed, while make always execute both. doit dry-run assumes that when a task is executed it would generate a different target than the cached one.

task_dep & uptodate

make has no concept of task_dep so nothing to compare to

dodo.py

def task_with_task_dep():
    yield {'name': 't1',
           'actions': ["echo hi1"],
           'file_dep': ['t1.in'],
       }
    yield {'name': 't2',
           'actions': ["echo hi2"],
           'uptodate': [lambda: True],
           'task_dep': ['with_task_dep:t1'],
       }

$ echo aaa > t1.in
$ doit with_task_dep
.  with_task_dep:t1
.  with_task_dep:t2
$ doit with_task_dep
-- with_task_dep:t1
-- with_task_dep:t2
$ doit --dry-run with_task_dep
-- with_task_dep:t1
-- with_task_dep:t2
$ echo xxx > t1.in
$ doit --dry-run with_task_dep
.  with_task_dep:t1
-- with_task_dep:t2

use case	expected	doit
task_dep needs execution	run	up-to-date

task_dep is also used for purely ordering purpose, in this case the expected output would be up-to-date.

Since as of now there is no way to distinguish the 2 use cases it is safer to assume task_dep really means the uptodate check of t2 would fail.

task_dep (without uptodate)

TODO

setup-task

TODO

calc_dep

TODO

--continue & --keep-going

In make there is a strong relation between --dry-run and --keep-going. If rule A change causes B to run on dry-run it means that --keep-going would not execute B if A fails.

doit already implements --continue. --dry-run and --continue should hold the same equivalence of --dry-run and --keep-going on make.