-
-
Save ProGamerGov/8f1d07d866700b159c4b15c4e2fda868 to your computer and use it in GitHub Desktop.
GPU Memory Usage In Neural-Style with 8 Nvidia Tesla K80 graphics cards
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Content Image size: 1051x976 | |
-content_layers relu4_2 | |
-style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 | |
-optimizer lbfgs (unless specified) | |
No -cudnn_autotune (unless specified) | |
Each Nvidia Tesla K80 graphics card used, has 11439MiB of Memory. | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2115 C /home/ubuntu/torch/install/bin/luajit 6491MiB | | |
| 1 2115 C /home/ubuntu/torch/install/bin/luajit 1763MiB | | |
| 2 2115 C /home/ubuntu/torch/install/bin/luajit 2313MiB | | |
| 3 2115 C /home/ubuntu/torch/install/bin/luajit 1510MiB | | |
| 4 2115 C /home/ubuntu/torch/install/bin/luajit 977MiB | | |
| 5 2115 C /home/ubuntu/torch/install/bin/luajit 1527MiB | | |
| 6 2115 C /home/ubuntu/torch/install/bin/luajit 842MiB | | |
| 7 2115 C /home/ubuntu/torch/install/bin/luajit 2097MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 1536 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -cudnn_autotune | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2302 C /home/ubuntu/torch/install/bin/luajit 7589MiB | | |
| 1 2302 C /home/ubuntu/torch/install/bin/luajit 2042MiB | | |
| 2 2302 C /home/ubuntu/torch/install/bin/luajit 2685MiB | | |
| 3 2302 C /home/ubuntu/torch/install/bin/luajit 1743MiB | | |
| 4 2302 C /home/ubuntu/torch/install/bin/luajit 1117MiB | | |
| 5 2302 C /home/ubuntu/torch/install/bin/luajit 1762MiB | | |
| 6 2302 C /home/ubuntu/torch/install/bin/luajit 958MiB | | |
| 7 2302 C /home/ubuntu/torch/install/bin/luajit 2416MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 1664 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -cudnn_autotune | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2452 C /home/ubuntu/torch/install/bin/luajit 10042MiB | | |
| 1 2452 C /home/ubuntu/torch/install/bin/luajit 2665MiB | | |
| 2 2452 C /home/ubuntu/torch/install/bin/luajit 3515MiB | | |
| 3 2452 C /home/ubuntu/torch/install/bin/luajit 2261MiB | | |
| 4 2452 C /home/ubuntu/torch/install/bin/luajit 1428MiB | | |
| 5 2452 C /home/ubuntu/torch/install/bin/luajit 2280MiB | | |
| 6 2452 C /home/ubuntu/torch/install/bin/luajit 1218MiB | | |
| 7 2452 C /home/ubuntu/torch/install/bin/luajit 3121MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 1920 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -cudnn_autotune | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2599 C /home/ubuntu/torch/install/bin/luajit 11402MiB | | |
| 1 2599 C /home/ubuntu/torch/install/bin/luajit 3011MiB | | |
| 2 2599 C /home/ubuntu/torch/install/bin/luajit 3976MiB | | |
| 3 2599 C /home/ubuntu/torch/install/bin/luajit 2549MiB | | |
| 4 2599 C /home/ubuntu/torch/install/bin/luajit 1601MiB | | |
| 5 2599 C /home/ubuntu/torch/install/bin/luajit 2568MiB | | |
| 6 2599 C /home/ubuntu/torch/install/bin/luajit 1362MiB | | |
| 7 2599 C /home/ubuntu/torch/install/bin/luajit 3513MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2048 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2950 C /home/ubuntu/torch/install/bin/luajit 3317MiB | | |
| 1 2950 C /home/ubuntu/torch/install/bin/luajit 4181MiB | | |
| 2 2950 C /home/ubuntu/torch/install/bin/luajit 5537MiB | | |
| 3 2950 C /home/ubuntu/torch/install/bin/luajit 3525MiB | | |
| 4 2950 C /home/ubuntu/torch/install/bin/luajit 2186MiB | | |
| 5 2950 C /home/ubuntu/torch/install/bin/luajit 3543MiB | | |
| 6 2950 C /home/ubuntu/torch/install/bin/luajit 1849MiB | | |
| 7 2950 C /home/ubuntu/torch/install/bin/luajit 5226MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2432 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -optimizer adam -cudnn_autotune | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3104 C /home/ubuntu/torch/install/bin/luajit 3657MiB | | |
| 1 3104 C /home/ubuntu/torch/install/bin/luajit 4616MiB | | |
| 2 3104 C /home/ubuntu/torch/install/bin/luajit 6116MiB | | |
| 3 3104 C /home/ubuntu/torch/install/bin/luajit 3887MiB | | |
| 4 3104 C /home/ubuntu/torch/install/bin/luajit 2404MiB | | |
| 5 3104 C /home/ubuntu/torch/install/bin/luajit 3906MiB | | |
| 6 3104 C /home/ubuntu/torch/install/bin/luajit 2031MiB | | |
| 7 3104 C /home/ubuntu/torch/install/bin/luajit 5341MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2560 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -optimizer adam -cudnn_autotune | |
All the loss values for every iteration are the same? | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3314 C /home/ubuntu/torch/install/bin/luajit 4389MiB | | |
| 1 3314 C /home/ubuntu/torch/install/bin/luajit 5553MiB | | |
| 2 3314 C /home/ubuntu/torch/install/bin/luajit 7365MiB | | |
| 3 3314 C /home/ubuntu/torch/install/bin/luajit 4668MiB | | |
| 4 3314 C /home/ubuntu/torch/install/bin/luajit 2873MiB | | |
| 5 3314 C /home/ubuntu/torch/install/bin/luajit 4686MiB | | |
| 6 3314 C /home/ubuntu/torch/install/bin/luajit 2421MiB | | |
| 7 3314 C /home/ubuntu/torch/install/bin/luajit 6398MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2816 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 -optimizer adam -cudnn_autotune |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Content Image size: 1920x1200 | |
-content_layers relu4_2 | |
-style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 | |
-optimizer adam | |
No -cudnn_autotune | |
Each Nvidia Tesla K80 graphics card used, has 11439MiB of Memory. | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2389 C /home/ubuntu/torch/install/bin/luajit 3724MiB | | |
| 1 2389 C /home/ubuntu/torch/install/bin/luajit 4599MiB | | |
| 2 2389 C /home/ubuntu/torch/install/bin/luajit 6307MiB | | |
| 3 2389 C /home/ubuntu/torch/install/bin/luajit 3240MiB | | |
| 4 2389 C /home/ubuntu/torch/install/bin/luajit 4007MiB | | |
| 5 2389 C /home/ubuntu/torch/install/bin/luajit 9920MiB | | |
| 6 2389 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
| 7 2389 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3000 -gpu 0,1,2,3,4,5 -multigpu_strategy 2,3,4,5,6 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2499 C /home/ubuntu/torch/install/bin/luajit 8179MiB | | |
| 1 2499 C /home/ubuntu/torch/install/bin/luajit 4007MiB | | |
| 2 2499 C /home/ubuntu/torch/install/bin/luajit 2476MiB | | |
| 3 2499 C /home/ubuntu/torch/install/bin/luajit 2379MiB | | |
| 4 2499 C /home/ubuntu/torch/install/bin/luajit 3244MiB | | |
| 5 2499 C /home/ubuntu/torch/install/bin/luajit 1707MiB | | |
| 6 2499 C /home/ubuntu/torch/install/bin/luajit 2090MiB | | |
| 7 2499 C /home/ubuntu/torch/install/bin/luajit 5395MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3000 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 4,6,8,9,10,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Memory Error | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3584 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 4,6,8,9,10,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2806 C /home/ubuntu/torch/install/bin/luajit 7213MiB | | |
| 1 2806 C /home/ubuntu/torch/install/bin/luajit 10022MiB | | |
| 2 2806 C /home/ubuntu/torch/install/bin/luajit 3458MiB | | |
| 3 2806 C /home/ubuntu/torch/install/bin/luajit 3326MiB | | |
| 4 2806 C /home/ubuntu/torch/install/bin/luajit 4553MiB | | |
| 5 2806 C /home/ubuntu/torch/install/bin/luajit 2361MiB | | |
| 6 2806 C /home/ubuntu/torch/install/bin/luajit 2909MiB | | |
| 7 2806 C /home/ubuntu/torch/install/bin/luajit 7579MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3584 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 3,6,8,9,10,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2953 C /home/ubuntu/torch/install/bin/luajit 7213MiB | | |
| 1 2953 C /home/ubuntu/torch/install/bin/luajit 8927MiB | | |
| 2 2953 C /home/ubuntu/torch/install/bin/luajit 7834MiB | | |
| 3 2953 C /home/ubuntu/torch/install/bin/luajit 3326MiB | | |
| 4 2953 C /home/ubuntu/torch/install/bin/luajit 4553MiB | | |
| 5 2953 C /home/ubuntu/torch/install/bin/luajit 2361MiB | | |
| 6 2953 C /home/ubuntu/torch/install/bin/luajit 2909MiB | | |
| 7 2953 C /home/ubuntu/torch/install/bin/luajit 7579MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3584 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 3,5,8,9,10,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3094 C /home/ubuntu/torch/install/bin/luajit 5239MiB | | |
| 1 3094 C /home/ubuntu/torch/install/bin/luajit 10903MiB | | |
| 2 3094 C /home/ubuntu/torch/install/bin/luajit 5644MiB | | |
| 3 3094 C /home/ubuntu/torch/install/bin/luajit 3458MiB | | |
| 4 3094 C /home/ubuntu/torch/install/bin/luajit 5547MiB | | |
| 5 3094 C /home/ubuntu/torch/install/bin/luajit 2361MiB | | |
| 6 3094 C /home/ubuntu/torch/install/bin/luajit 2909MiB | | |
| 7 3094 C /home/ubuntu/torch/install/bin/luajit 7579MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3584 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,4,6,8,10,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3246 C /home/ubuntu/torch/install/bin/luajit 5239MiB | | |
| 1 3246 C /home/ubuntu/torch/install/bin/luajit 6496MiB | | |
| 2 3246 C /home/ubuntu/torch/install/bin/luajit 8927MiB | | |
| 3 3246 C /home/ubuntu/torch/install/bin/luajit 5644MiB | | |
| 4 3246 C /home/ubuntu/torch/install/bin/luajit 3458MiB | | |
| 5 3246 C /home/ubuntu/torch/install/bin/luajit 5547MiB | | |
| 6 3246 C /home/ubuntu/torch/install/bin/luajit 2909MiB | | |
| 7 3246 C /home/ubuntu/torch/install/bin/luajit 7579MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3584 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3355 C /home/ubuntu/torch/install/bin/luajit 5989MiB | | |
| 1 3355 C /home/ubuntu/torch/install/bin/luajit 7434MiB | | |
| 2 3355 C /home/ubuntu/torch/install/bin/luajit 10224MiB | | |
| 3 3355 C /home/ubuntu/torch/install/bin/luajit 6454MiB | | |
| 4 3355 C /home/ubuntu/torch/install/bin/luajit 3944MiB | | |
| 5 3355 C /home/ubuntu/torch/install/bin/luajit 6340MiB | | |
| 6 3355 C /home/ubuntu/torch/install/bin/luajit 3314MiB | | |
| 7 3355 C /home/ubuntu/torch/install/bin/luajit 8657MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 3840 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3595 C /home/ubuntu/torch/install/bin/luajit 6484MiB | | |
| 1 3595 C /home/ubuntu/torch/install/bin/luajit 8053MiB | | |
| 2 3595 C /home/ubuntu/torch/install/bin/luajit 11078MiB | | |
| 3 3595 C /home/ubuntu/torch/install/bin/luajit 6989MiB | | |
| 4 3595 C /home/ubuntu/torch/install/bin/luajit 4265MiB | | |
| 5 3595 C /home/ubuntu/torch/install/bin/luajit 6865MiB | | |
| 6 3595 C /home/ubuntu/torch/install/bin/luajit 3581MiB | | |
| 7 3595 C /home/ubuntu/torch/install/bin/luajit 9378MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 4000 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3688 C /home/ubuntu/torch/install/bin/luajit 6586MiB | | |
| 1 3688 C /home/ubuntu/torch/install/bin/luajit 8180MiB | | |
| 2 3688 C /home/ubuntu/torch/install/bin/luajit 11254MiB | | |
| 3 3688 C /home/ubuntu/torch/install/bin/luajit 7098MiB | | |
| 4 3688 C /home/ubuntu/torch/install/bin/luajit 4330MiB | | |
| 5 3688 C /home/ubuntu/torch/install/bin/luajit 6971MiB | | |
| 6 3688 C /home/ubuntu/torch/install/bin/luajit 3636MiB | | |
| 7 3688 C /home/ubuntu/torch/install/bin/luajit 9521MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 4032 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 2,3,4,6,8,11,12 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 2535 C /home/ubuntu/torch/install/bin/luajit 4672MiB | | |
| 1 2535 C /home/ubuntu/torch/install/bin/luajit 6207MiB | | |
| 2 2535 C /home/ubuntu/torch/install/bin/luajit 4550MiB | | |
| 3 2535 C /home/ubuntu/torch/install/bin/luajit 1534MiB | | |
| 4 2535 C /home/ubuntu/torch/install/bin/luajit 2364MiB | | |
| 5 2535 C /home/ubuntu/torch/install/bin/luajit 1564MiB | | |
| 6 2535 C /home/ubuntu/torch/install/bin/luajit 1051MiB | | |
| 7 2535 C /home/ubuntu/torch/install/bin/luajit 288MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2432 -gpu 0,1,2,3,4,5,6,7 -multigpu_strategy 3,6,12,15,20,26,31 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3497 C /home/ubuntu/torch/install/bin/luajit 4672MiB | | |
| 1 3497 C /home/ubuntu/torch/install/bin/luajit 6207MiB | | |
| 2 3497 C /home/ubuntu/torch/install/bin/luajit 4550MiB | | |
| 3 3497 C /home/ubuntu/torch/install/bin/luajit 1534MiB | | |
| 4 3497 C /home/ubuntu/torch/install/bin/luajit 2364MiB | | |
| 5 3497 C /home/ubuntu/torch/install/bin/luajit 1564MiB | | |
| 6 3497 C /home/ubuntu/torch/install/bin/luajit 1112MiB | | |
| 7 3497 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
+-----------------------------------------------------------------------------+ | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 3732 C /home/ubuntu/torch/install/bin/luajit 4672MiB | | |
| 1 3732 C /home/ubuntu/torch/install/bin/luajit 6207MiB | | |
| 2 3732 C /home/ubuntu/torch/install/bin/luajit 4550MiB | | |
| 3 3732 C /home/ubuntu/torch/install/bin/luajit 1534MiB | | |
| 4 3732 C /home/ubuntu/torch/install/bin/luajit 2364MiB | | |
| 5 3732 C /home/ubuntu/torch/install/bin/luajit 2151MiB | | |
| 6 3732 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
| 7 3732 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2432 -gpu 0,1,2,3,4,5 -multigpu_strategy 3,6,12,15,20 | |
+-----------------------------------------------------------------------------+ | |
| Processes: GPU Memory | | |
| GPU PID Type Process name Usage | | |
|=============================================================================| | |
| 0 4107 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
| 1 4107 C /home/ubuntu/torch/install/bin/luajit 8025MiB | | |
| 2 4107 C /home/ubuntu/torch/install/bin/luajit 4550MiB | | |
| 3 4107 C /home/ubuntu/torch/install/bin/luajit 1534MiB | | |
| 4 4107 C /home/ubuntu/torch/install/bin/luajit 2364MiB | | |
| 5 4107 C /home/ubuntu/torch/install/bin/luajit 1564MiB | | |
| 6 4107 C /home/ubuntu/torch/install/bin/luajit 1112MiB | | |
| 7 4107 C /home/ubuntu/torch/install/bin/luajit 145MiB | | |
+-----------------------------------------------------------------------------+ | |
-image_size 2432 -gpu 1,2,3,4,5,6 -multigpu_strategy 6,12,15,20,26 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment