Skip to content

Instantly share code, notes, and snippets.

@jramapuram
Created August 15, 2020 12:32
Show Gist options
  • Save jramapuram/d2feb0bb57321c29a0038db289f77e5b to your computer and use it in GitHub Desktop.
Save jramapuram/d2feb0bb57321c29a0038db289f77e5b to your computer and use it in GitHub Desktop.
2020-08-15 01:10:54,597 DEBUG MainThread:141411 [wandb_config.py:_load_defaults():154] no defaults not found in config-defaults.yaml
2020-08-15 01:10:54,747 DEBUG MainThread:141411 [meta.py:_setup_code_git():49] probe for git information
2020-08-15 01:10:54,986 DEBUG MainThread:141411 [meta.py:setup():104] code probe starting
2020-08-15 01:10:54,987 DEBUG MainThread:141411 [meta.py:_setup_code_program():58] save program starting
2020-08-15 01:10:54,987 DEBUG MainThread:141411 [meta.py:_setup_code_program():60] save program starting: /home/jramapuram/sshfs/kanerva_plus_plus/./main.py
2020-08-15 01:10:54,999 DEBUG MainThread:141411 [meta.py:_setup_code_program():68] save program saved: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/code/main.py
2020-08-15 01:10:55,003 DEBUG MainThread:141411 [meta.py:_setup_code_program():70] save program
2020-08-15 01:10:55,036 DEBUG MainThread:141411 [meta.py:setup():124] code probe done
2020-08-15 01:10:55,084 DEBUG MainThread:141411 [run_manager.py:__init__():547] Initialized sync for kanerva_plus_plus/2kcpmyzj
2020-08-15 01:10:55,102 INFO MainThread:141411 [run_manager.py:wrap_existing_process():1149] wrapping existing process 141328
2020-08-15 01:10:55,103 WARNING MainThread:141411 [io_wrap.py:register():108] SIGWINCH handler was not None: <Handlers.SIG_DFL: 0>
2020-08-15 01:10:55,247 INFO MainThread:141411 [run_manager.py:init_run():928] system metrics and metadata threads started
2020-08-15 01:10:55,248 INFO MainThread:141411 [run_manager.py:init_run():967] upserting run before process can begin, waiting at most 10 seconds
2020-08-15 01:10:55,389 INFO Thread-15 :141411 [run_manager.py:_upsert_run():1052] saving patches
2020-08-15 01:10:55,762 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/config.yaml
2020-08-15 01:11:04,313 ERROR Thread-15 :141411 [internal.py:save_patches():210] Error generating diff: Command '['git', 'diff', '--submodule=diff', '3d9e8cf427f8b8d1868fa378e64a5ca4c8afdfe9']' timed out after 5 seconds
2020-08-15 01:11:04,314 INFO Thread-15 :141411 [run_manager.py:_upsert_run():1056] saving pip packages
2020-08-15 01:11:04,334 INFO Thread-15 :141411 [run_manager.py:_upsert_run():1058] initializing streaming files api
2020-08-15 01:11:04,336 INFO Thread-15 :141411 [run_manager.py:_upsert_run():1065] unblocking file change observer, beginning sync with W&B servers
2020-08-15 01:11:04,337 INFO MainThread:141411 [run_manager.py:wrap_existing_process():1166] informing user process we are ready to proceed
2020-08-15 01:11:04,344 INFO MainThread:141411 [run_manager.py:_sync_etc():1298] entering loop for messages from user process
2020-08-15 01:11:04,398 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:04,399 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-metadata.json
2020-08-15 01:11:04,399 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/code/main.py
2020-08-15 01:11:04,400 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/code
2020-08-15 01:11:04,400 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/diff.patch
2020-08-15 01:11:04,401 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/diff.patch
2020-08-15 01:11:04,401 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/upstream_diff_3d9e8cf427f8b8d1868fa378e64a5ca4c8afdfe9.patch
2020-08-15 01:11:04,406 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/upstream_diff_3d9e8cf427f8b8d1868fa378e64a5ca4c8afdfe9.patch
2020-08-15 01:11:04,831 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/upstream_diff_3d9e8cf427f8b8d1868fa378e64a5ca4c8afdfe9.patch
2020-08-15 01:11:04,832 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:04,833 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/config.yaml
2020-08-15 01:11:04,834 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/requirements.txt
2020-08-15 01:11:11,847 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-metadata.json
2020-08-15 01:11:12,880 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:12,881 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/graph/graph_0_summary_28e98832.graph.json
2020-08-15 01:11:12,882 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:12,911 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/graph
2020-08-15 01:11:12,912 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media
2020-08-15 01:11:14,892 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:15,901 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:17,914 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:19,942 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:19,943 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_read-traces-minibatch0_1_c5108def.png
2020-08-15 01:11:19,944 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_input_1_683fe2ec.png
2020-08-15 01:11:19,945 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_memory_1_49033288.png
2020-08-15 01:11:19,946 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_reconstruction_1_e4b6f0d0.png
2020-08-15 01:11:19,947 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media
2020-08-15 01:11:19,948 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:20,960 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-history.jsonl
2020-08-15 01:11:20,963 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_reconstruction_1_ed252709.png
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mean-perturbed-generated_1_491d352c.png
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_input_1_5a4bb46f.png
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_memory_1_4c75315f.png
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mosiac-generated_1_c061bdfe.png
2020-08-15 01:11:20,974 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_read-traces-minibatch0_1_12312dca.png
2020-08-15 01:11:20,975 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mean-perturbed-generated_1_043b279d.png
2020-08-15 01:11:20,975 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mosiac-generated_1_4befb5d2.png
2020-08-15 01:11:20,975 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:23,976 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:23,977 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-events.jsonl
2020-08-15 01:11:25,996 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-history.jsonl
2020-08-15 01:11:25,997 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:25,997 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:26,005 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_read-traces-minibatch0_2_bc2cdc4d.png
2020-08-15 01:11:26,005 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mosiac-generated_2_3c8dc0f9.png
2020-08-15 01:11:26,005 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_input_2_f4d6518c.png
2020-08-15 01:11:26,006 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_memory_2_e1a020e8.png
2020-08-15 01:11:26,006 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_reconstruction_2_d6b38c13.png
2020-08-15 01:11:26,006 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_input_2_5a4bb46f.png
2020-08-15 01:11:26,006 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mean-perturbed-generated_2_c414b854.png
2020-08-15 01:11:26,006 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mean-perturbed-generated_2_7e71184e.png
2020-08-15 01:11:26,007 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_read-traces-minibatch0_2_6afea02d.png
2020-08-15 01:11:26,007 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_memory_2_061862ff.png
2020-08-15 01:11:26,007 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mosiac-generated_2_39566a9b.png
2020-08-15 01:11:26,007 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_reconstruction_2_85dd3043.png
2020-08-15 01:11:26,007 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:28,007 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-metadata.json
2020-08-15 01:11:29,013 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:31,039 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-history.jsonl
2020-08-15 01:11:31,040 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:31,040 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:31,049 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_input_3_5a4bb46f.png
2020-08-15 01:11:31,049 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_read-traces-minibatch0_3_10a1678f.png
2020-08-15 01:11:31,049 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_reconstruction_3_4dc87bf9.png
2020-08-15 01:11:31,049 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_read-traces-minibatch0_3_b62e3529.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_memory_3_2ece532a.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_reconstruction_3_7b70b2f2.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mosiac-generated_3_4999ff5a.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mosiac-generated_3_586a2788.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mean-perturbed-generated_3_f9f9bb79.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_memory_3_0de8d7ab.png
2020-08-15 01:11:31,050 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_input_3_83960a2f.png
2020-08-15 01:11:31,051 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mean-perturbed-generated_3_cb126348.png
2020-08-15 01:11:31,051 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:34,056 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:36,101 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/output.log
2020-08-15 01:11:36,102 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_reconstruction_4_dd881ea5.png
2020-08-15 01:11:36,103 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_input_4_098748ce.png
2020-08-15 01:11:36,104 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_reconstruction_4_b997b6a7.png
2020-08-15 01:11:36,105 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_memory_4_7a43be3a.png
2020-08-15 01:11:36,107 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mean-perturbed-generated_4_58355424.png
2020-08-15 01:11:36,108 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_read-traces-minibatch0_4_b58a9e29.png
2020-08-15 01:11:36,109 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_memory_4_2380023d.png
2020-08-15 01:11:36,110 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_mosiac-generated_4_851c2d7c.png
2020-08-15 01:11:36,111 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/train_read-traces-minibatch0_4_1a136185.png
2020-08-15 01:11:36,112 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:36,644 INFO MainThread:141411 [run_manager.py:_sync_etc():1355] process received interrupt signal, shutting down
2020-08-15 01:11:36,645 INFO MainThread:141411 [run_manager.py:_sync_etc():1408] closing log streams and sending exitcode to W&B
2020-08-15 01:11:36,645 INFO MainThread:141411 [run_manager.py:shutdown():1072] shutting down system stats and metadata service
2020-08-15 01:11:37,135 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-history.jsonl
2020-08-15 01:11:37,136 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-events.jsonl
2020-08-15 01:11:37,137 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-summary.json
2020-08-15 01:11:37,146 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mean-perturbed-generated_4_d58e4dd5.png
2020-08-15 01:11:37,147 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_input_4_5a4bb46f.png
2020-08-15 01:11:37,148 INFO Thread-4 :141411 [run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mosiac-generated_4_437fed77.png
2020-08-15 01:11:37,150 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:37,336 INFO MainThread:141411 [run_manager.py:shutdown():1086] stopping streaming files and file change observer
2020-08-15 01:11:38,149 INFO Thread-4 :141411 [run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-metadata.json
2020-08-15 01:11:40,034 INFO MainThread:141411 [run_manager.py:_sync_etc():1427] rendering summary
2020-08-15 01:11:40,045 INFO MainThread:141411 [run_manager.py:_sync_etc():1459] syncing files to cloud storage
2020-08-15 01:11:40,300 INFO MainThread:141411 [run_manager.py:_sync_etc():1468] syncing complete: http://devbox:8080/jramapuram/kanerva_plus_plus/runs/2kcpmyzj
8 INFO Thread-4 :141411 [2kcpmyzj:run_manager.py:_on_file_created():681] file/dir created: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images/test_mosiac-generated_4_437fed77.png
2020-08-15 01:11:37,150 INFO Thread-4 :141411 [2kcpmyzj:run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/media/images
2020-08-15 01:11:37,336 INFO MainThread:141411 [2kcpmyzj:run_manager.py:shutdown():1086] stopping streaming files and file change observer
2020-08-15 01:11:38,149 INFO Thread-4 :141411 [2kcpmyzj:run_manager.py:_on_file_modified():692] file/dir modified: /home/jramapuram/sshfs/kanerva_plus_plus/wandb/run-20200815_011054-2kcpmyzj/wandb-metadata.json
2020-08-15 01:11:40,034 INFO MainThread:141411 [2kcpmyzj:run_manager.py:_sync_etc():1427] rendering summary
2020-08-15 01:11:40,045 INFO MainThread:141411 [2kcpmyzj:run_manager.py:_sync_etc():1459] syncing files to cloud storage
2020-08-15 01:11:40,300 INFO MainThread:141411 [2kcpmyzj:run_manager.py:_sync_etc():1468] syncing complete: http://devbox:8080/jramapuram/kanerva_plus_plus/runs/2kcpmyzj
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment