Created
September 10, 2010 13:32
-
-
Save chrisgorgo/573621 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
filo@filo-dtc:~/workspace/nipype$ git checkout 8c388d3262b1b3eedf79 | |
Note: checking out '8c388d3262b1b3eedf79'. | |
You are in 'detached HEAD' state. You can look around, make experimental | |
changes and commit them, and you can discard any commits you make in this | |
state without impacting any branches by performing another checkout. | |
If you want to create a new branch to retain commits you create, you may | |
do so (now or later) by using -b with the checkout command again. Example: | |
git checkout -b new_branch_name | |
HEAD is now at 8c388d3... changing the number of residual saved in defaults | |
filo@filo-dtc:/media/sdb2/nipype_examples$ python ~/workspace/nipype/examples/bootstrapping.py | |
/usr/lib/pymodules/python2.6/IPython/kernel/clientconnector.py:43: DeprecationWarning: Importing class Tub directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub instead | |
self.tub = Tub() | |
DEBUG:workflow:(preproc.realign, preproc.coregister): No edge data | |
DEBUG:workflow:(preproc.realign, preproc.coregister): new edge data: {'connect': [('mean_image', 'source'), ('realigned_files', 'apply_to_files')]} | |
DEBUG:workflow:(preproc.coregister, preproc.smooth): No edge data | |
DEBUG:workflow:(preproc.coregister, preproc.smooth): new edge data: {'connect': [('coregistered_files', 'in_files')]} | |
DEBUG:workflow:(preproc.realign, preproc.art): No edge data | |
DEBUG:workflow:(preproc.realign, preproc.art): new edge data: {'connect': [('realignment_parameters', 'realignment_parameters')]} | |
DEBUG:workflow:(preproc.coregister, preproc.art): No edge data | |
DEBUG:workflow:(preproc.coregister, preproc.art): new edge data: {'connect': [('coregistered_files', 'realigned_files')]} | |
DEBUG:workflow:(detrend.modelspecd, detrend.level1designd): No edge data | |
DEBUG:workflow:(detrend.modelspecd, detrend.level1designd): new edge data: {'connect': [('session_info', 'session_info')]} | |
DEBUG:workflow:(detrend.level1designd, detrend.level1estimated): No edge data | |
DEBUG:workflow:(detrend.level1designd, detrend.level1estimated): new edge data: {'connect': [('spm_mat_file', 'spm_mat_file')]} | |
DEBUG:workflow:(detrend.level1estimated, detrend.join_residuals): No edge data | |
DEBUG:workflow:(detrend.level1estimated, detrend.join_residuals): new edge data: {'connect': [('residual_images', 'in_files')]} | |
DEBUG:workflow:(detrend.join_residuals, detrend.img2float): No edge data | |
DEBUG:workflow:(detrend.join_residuals, detrend.img2float): new edge data: {'connect': [('merged_file', 'in_file')]} | |
DEBUG:workflow:(analysis.modelspec, analysis.level1design): No edge data | |
DEBUG:workflow:(analysis.modelspec, analysis.level1design): new edge data: {'connect': [('session_info', 'session_info')]} | |
DEBUG:workflow:(analysis.level1design, analysis.level1estimate): No edge data | |
DEBUG:workflow:(analysis.level1design, analysis.level1estimate): new edge data: {'connect': [('spm_mat_file', 'spm_mat_file')]} | |
DEBUG:workflow:(analysis.level1estimate, analysis.contrastestimate): No edge data | |
DEBUG:workflow:(analysis.level1estimate, analysis.contrastestimate): new edge data: {'connect': [('spm_mat_file', 'spm_mat_file'), ('beta_images', 'beta_images'), ('mean_residual_image', 'mean_residual_image')]} | |
DEBUG:workflow:(analysis.contrastestimate, analysis.threshold): No edge data | |
DEBUG:workflow:(analysis.contrastestimate, analysis.threshold): new edge data: {'connect': [('spm_mat_file', 'spm_mat_file'), ('spmT_images', 'spmT_images')]} | |
DEBUG:workflow:(analysis.level1estimate, analysis.threshold): No edge data | |
DEBUG:workflow:(analysis.level1estimate, analysis.threshold): new edge data: {'connect': [('RPVimage', 'RPVimage'), ('mask_image', 'mask_image'), ('beta_images', 'beta_images'), ('mean_residual_image', 'mean_residual_image')]} | |
DEBUG:workflow:(bootstrap_wf.detrend, bootstrap_wf.bootstrap): No edge data | |
DEBUG:workflow:(bootstrap_wf.detrend, bootstrap_wf.bootstrap): new edge data: {'connect': [('img2float.out_file', 'original_volume')]} | |
DEBUG:workflow:(bootstrap_wf.bootstrap, bootstrap_wf.analysis): No edge data | |
DEBUG:workflow:(bootstrap_wf.bootstrap, bootstrap_wf.analysis): new edge data: {'connect': [('bootstraped_volume', 'modelspec.functional_runs')]} | |
DEBUG:workflow:(bootstrap_wf.detrend, bootstrap_wf.analysis): No edge data | |
DEBUG:workflow:(bootstrap_wf.detrend, bootstrap_wf.analysis): new edge data: {'connect': [('level1estimated.mask_image', 'level1design.mask_image')]} | |
DEBUG:workflow:(level1.infosource, level1.datasource): No edge data | |
DEBUG:workflow:(level1.infosource, level1.datasource): new edge data: {'connect': [('subject_id', 'subject_id')]} | |
DEBUG:workflow:(level1.datasource, level1.preproc): No edge data | |
DEBUG:workflow:(level1.datasource, level1.preproc): new edge data: {'connect': [('func', 'realign.in_files')]} | |
DEBUG:workflow:(level1.datasource, level1.preproc): Edge data exists: {'connect': [('func', 'realign.in_files')]} | |
DEBUG:workflow:(level1.datasource, level1.preproc): new edge data: {'connect': [('func', 'realign.in_files'), ('struct', 'coregister.target')]} | |
DEBUG:workflow:(level1.preproc, level1.bootstrap_wf): No edge data | |
DEBUG:workflow:(level1.preproc, level1.bootstrap_wf): new edge data: {'connect': [('realign.realignment_parameters', 'detrend.modelspecd.realignment_parameters'), ('art.outlier_files', 'detrend.modelspecd.outlier_files'), ('smooth.smoothed_files', 'detrend.modelspecd.functional_runs')]} | |
DEBUG:workflow:setting nodelevel input concatenate_runs = True | |
DEBUG:workflow:setting nodelevel input input_units = secs | |
DEBUG:workflow:setting nodelevel input output_units = secs | |
DEBUG:workflow:setting nodelevel input time_repetition = 3.0 | |
DEBUG:workflow:setting nodelevel input high_pass_filter_cutoff = 120.0 | |
DEBUG:workflow:setting nodelevel input subject_info = [Bunch(amplitudes=None, conditions=[], durations=[], onsets=[], pmod=None, regressor_names=None, regressors=None, tmod=None), Bunch(amplitudes=None, conditions=[], durations=[], onsets=[], pmod=None, regressor_names=None, regressors=None, tmod=None), Bunch(amplitudes=None, conditions=[], durations=[], onsets=[], pmod=None, regressor_names=None, regressors=None, tmod=None), Bunch(amplitudes=None, conditions=[], durations=[], onsets=[], pmod=None, regressor_names=None, regressors=None, tmod=None)] | |
DEBUG:workflow:setting nodelevel input interscan_interval = 3.0 | |
DEBUG:workflow:setting nodelevel input timing_units = secs | |
DEBUG:workflow:setting nodelevel input blocks_info = Bunch(duration=[10, 10], onsets=[[5, 25, 45, 65, 90, 110, 130, 150, 175, 195, 215, 235, 260, 280, 300, 320], [15, 35, 55, 75, 100, 120, 140, 160, 185, 205, 225, 245, 270, 290, 310, 330]]) | |
DEBUG:workflow:setting nodelevel input time_repetition = 3.0 | |
DEBUG:workflow:setting nodelevel input subject_info = [Bunch(amplitudes=None, conditions=['Task-Odd', 'Task-Even'], durations=[[5], [5]], onsets=[[5, 25, 45, 65, 90, 110, 130, 150, 175, 195, 215, 235, 260, 280, 300, 320], [15, 35, 55, 75, 100, 120, 140, 160, 185, 205, 225, 245, 270, 290, 310, 330]], pmod=None, regressor_names=None, regressors=None, tmod=None)] | |
DEBUG:workflow:setting nodelevel input input_units = scans | |
DEBUG:workflow:setting nodelevel input output_units = scans | |
DEBUG:workflow:setting nodelevel input interscan_interval = 3.0 | |
DEBUG:workflow:setting nodelevel input timing_units = secs | |
DEBUG:workflow:setting nodelevel input contrasts = [('Task>Baseline', 'T', ['Task-Odd', 'Task-Even'], [0.5, 0.5]), ('Task-Odd>Task-Even', 'T', ['Task-Odd', 'Task-Even'], [1.0, -1.0])] | |
DEBUG:workflow:setting nodelevel input contrast_index = 1 | |
DEBUG:workflow:(level2.l2source, level2.frequency_map): No edge data | |
DEBUG:workflow:(level2.l2source, level2.frequency_map): new edge data: {'connect': [('outfiles', 'binary_images')]} | |
DEBUG:workflow:(level2.frequency_map, level2.simple_threshold): No edge data | |
DEBUG:workflow:(level2.frequency_map, level2.simple_threshold): new edge data: {'connect': [('frequency_map', 'volumes')]} | |
DEBUG:workflow:in: ('realign.realignment_parameters', 'detrend.modelspecd.realignment_parameters') | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): No edge data | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): new edge data: {'connect': [('realign.realignment_parameters', 'realignment_parameters')]} | |
DEBUG:workflow:in: ('art.outlier_files', 'detrend.modelspecd.outlier_files') | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): Edge data exists: {'connect': [('realign.realignment_parameters', 'realignment_parameters')]} | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): new edge data: {'connect': [('realign.realignment_parameters', 'realignment_parameters'), ('art.outlier_files', 'outlier_files')]} | |
DEBUG:workflow:in: ('smooth.smoothed_files', 'detrend.modelspecd.functional_runs') | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): Edge data exists: {'connect': [('realign.realignment_parameters', 'realignment_parameters'), ('art.outlier_files', 'outlier_files')]} | |
DEBUG:workflow:(level1.preproc, detrend.modelspecd): new edge data: {'connect': [('realign.realignment_parameters', 'realignment_parameters'), ('art.outlier_files', 'outlier_files'), ('smooth.smoothed_files', 'functional_runs')]} | |
DEBUG:workflow:out: ('img2float.out_file', 'original_volume') | |
DEBUG:workflow:(detrend.img2float, bootstrap_wf.bootstrap): No edge data | |
DEBUG:workflow:(detrend.img2float, bootstrap_wf.bootstrap): new edge data: {'connect': [('out_file', 'original_volume')]} | |
DEBUG:workflow:out: ('level1estimated.mask_image', 'level1design.mask_image') | |
DEBUG:workflow:(detrend.level1estimated, bootstrap_wf.analysis): No edge data | |
DEBUG:workflow:(detrend.level1estimated, bootstrap_wf.analysis): new edge data: {'connect': [('mask_image', 'level1design.mask_image')]} | |
DEBUG:workflow:in: ('bootstraped_volume', 'modelspec.functional_runs') | |
DEBUG:workflow:(bootstrap_wf.bootstrap, analysis.modelspec): No edge data | |
DEBUG:workflow:(bootstrap_wf.bootstrap, analysis.modelspec): new edge data: {'connect': [('bootstraped_volume', 'functional_runs')]} | |
DEBUG:workflow:in: ('level1estimated.mask_image', 'level1design.mask_image') | |
DEBUG:workflow:(bootstrap_wf.detrend, analysis.level1design): No edge data | |
DEBUG:workflow:(bootstrap_wf.detrend, analysis.level1design): new edge data: {'connect': [('level1estimated.mask_image', 'mask_image')]} | |
DEBUG:workflow:in: ('mask_image', 'level1design.mask_image') | |
DEBUG:workflow:(bootstrap_wf.detrend.level1estimated, analysis.level1design): No edge data | |
DEBUG:workflow:(bootstrap_wf.detrend.level1estimated, analysis.level1design): new edge data: {'connect': [('mask_image', 'mask_image')]} | |
DEBUG:workflow:in: ('func', 'realign.in_files') | |
DEBUG:workflow:(level1.datasource, preproc.realign): No edge data | |
DEBUG:workflow:(level1.datasource, preproc.realign): new edge data: {'connect': [('func', 'in_files')]} | |
DEBUG:workflow:in: ('struct', 'coregister.target') | |
DEBUG:workflow:(level1.datasource, preproc.coregister): No edge data | |
DEBUG:workflow:(level1.datasource, preproc.coregister): new edge data: {'connect': [('struct', 'target')]} | |
DEBUG:workflow:out: ('realign.realignment_parameters', 'detrend.modelspecd.realignment_parameters') | |
/home/filo/workspace/nipype/nipype/pipeline/engine.py:301: UserWarning: Module bootstrap_wf has no input called detrend.modelspecd.realignment_parameters | |
info[2])) | |
Traceback (most recent call last): | |
File "/home/filo/workspace/nipype/examples/bootstrapping.py", line 278, in <module> | |
l1pipeline.run() | |
File "/home/filo/workspace/nipype/nipype/pipeline/engine.py", line 400, in run | |
self._create_flat_graph() | |
File "/home/filo/workspace/nipype/nipype/pipeline/engine.py", line 511, in _create_flat_graph | |
workflowcopy._generate_execgraph() | |
File "/home/filo/workspace/nipype/nipype/pipeline/engine.py", line 554, in _generate_execgraph | |
self.connect(srcnode, srcout, dstnode, dstin) | |
File "/home/filo/workspace/nipype/nipype/pipeline/engine.py", line 303, in connect | |
raise Exception('Some connections were not found') | |
Exception: Some connections were not found |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Satrajit Ghosh 10:50
i know the problem
Chris Filo Gorgolewski 10:50
oh wow!
although I silently hoped for that... 10:51
Satrajit Ghosh 10:51
there is no particular order in which nested workflows are being expand
Chris Filo Gorgolewski 10:51
that's great
Satrajit Ghosh 10:51
so depending on which one gets expanded first and your set of connections, sometimes you see the error and sometimes you don't
Chris Filo Gorgolewski 10:52
still the order should not influence the result
Satrajit Ghosh 10:53
think of A(C)B(D) and now you have connections between C and D
if B(D) gets expanded first then C will connect appropriately 10:53
if A(C) gets expanded first then C will not have an appropriate connection. i thought we had addressed this, but i very much suspect that this is the root cause 10:54
Chris Filo Gorgolewski 10:54
by expanding you mean flattening and renaming?
Satrajit Ghosh 10:55
yes
there is actually a separate problem in expanding 10:55
Chris Filo Gorgolewski 10:55
I see the problem now
so shouldn't we sort them first? 10:56
Satrajit Ghosh 10:56
i don't think that handles all the cases
Satrajit Ghosh 11:02
but doing a topological sort at each workflow level is probably a good idea
nodes = nx.topological_sort(self._graph)
for node in nodes: 11:02
if you put those lines in _generate_execgraph 11:02
but i'm really not sure that's the answer 11:02
Chris Filo Gorgolewski 11:03
I'll try this
Satrajit Ghosh 11:03
it works for your case, but you need to create some test cases
if after the sort line you put nodes.reverse() 11:04
it will crash everytime 11:04
Chris Filo Gorgolewski 11:05
it should be at least deterministic...
even if we are not sure if it solves all of the cases 11:06
Satrajit Ghosh 11:08
i think doing the topological sort will address your case, but the following case will likely break
consider two workflows A->B and C->D 11:08
now consider D getting an input from the output of A and B getting an input from the output of C. i have a feeling the execgraph is going to break 11:09
Chris Filo Gorgolewski 11:12
cant we do a two stage expansion?
first flatten and rename and then reconnect? 11:12
Satrajit Ghosh 11:13
the key is to determine where a connection port (stored in an edge data) connects to.
anyway, i really need to get back to finishing a grant 11:13
Chris Filo Gorgolewski 11:13
sure
thanks for your help 11:14
Satrajit Ghosh 11:14
no problem
but we should fix this bug. 11:14