Skip to content

Instantly share code, notes, and snippets.

@cieldeville
Created April 23, 2023 15:05
Show Gist options
  • Save cieldeville/b5065befc319e801d14b0fcdd4d9f20a to your computer and use it in GitHub Desktop.
Save cieldeville/b5065befc319e801d14b0fcdd4d9f20a to your computer and use it in GitHub Desktop.
Tracemalloc memory allocations
Embedding words into BERT model...
Evaluated 0.00% % of words ( 0 / 2683 )
[ Top 10 Memory Allocations ]
<frozen importlib._bootstrap_external>:525: size=1378 KiB, count=8414, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\tokenization_bert.py:199: size=125 KiB, count=2000, average=64 B
C:\01_SDKs\02_Python\3_7_2\lib\dataclasses.py:386: size=120 KiB, count=1319, average=93 B
<frozen importlib._bootstrap_external>:1433: size=105 KiB, count=779, average=138 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\module.py:270: size=68.0 KiB, count=458, average=152 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\module.py:269: size=68.0 KiB, count=458, average=152 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\module.py:268: size=68.0 KiB, count=458, average=152 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\module.py:267: size=67.8 KiB, count=457, average=152 B
Evaluated 3.73% % of words ( 100 / 2683 )
[ Top 10 Memory Allocations ]
<frozen importlib._bootstrap_external>:525: size=1365 KiB, count=8311, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\linear.py:103: size=1043 KiB, count=13348, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:2486: size=544 KiB, count=6954, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:441: size=261 KiB, count=3336, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:363: size=261 KiB, count=3336, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:342: size=261 KiB, count=3336, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\activations.py:56: size=261 KiB, count=3336, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:1818: size=261 KiB, count=3336, average=80 B
Evaluated 7.45% % of words ( 200 / 2683 )
[ Top 10 Memory Allocations ]
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\linear.py:103: size=2089 KiB, count=26740, average=80 B
<frozen importlib._bootstrap_external>:525: size=1364 KiB, count=8300, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:2486: size=1089 KiB, count=13929, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:441: size=522 KiB, count=6684, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:363: size=522 KiB, count=6684, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:342: size=522 KiB, count=6684, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\activations.py:56: size=522 KiB, count=6684, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:1818: size=522 KiB, count=6684, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
Evaluated 11.18% % of words ( 300 / 2683 )
[ Top 10 Memory Allocations ]
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\linear.py:103: size=3098 KiB, count=39650, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:2486: size=1614 KiB, count=20654, average=80 B
<frozen importlib._bootstrap_external>:525: size=1364 KiB, count=8300, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:441: size=774 KiB, count=9912, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:363: size=774 KiB, count=9912, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:342: size=774 KiB, count=9912, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\activations.py:56: size=774 KiB, count=9912, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:1818: size=774 KiB, count=9912, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
Evaluated 14.91% % of words ( 400 / 2683 )
[ Top 10 Memory Allocations ]
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\linear.py:103: size=4024 KiB, count=51506, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:2486: size=2096 KiB, count=26829, average=80 B
<frozen importlib._bootstrap_external>:525: size=1364 KiB, count=8300, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:441: size=1006 KiB, count=12876, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:363: size=1006 KiB, count=12876, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:342: size=1006 KiB, count=12876, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\activations.py:56: size=1006 KiB, count=12876, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:1818: size=1006 KiB, count=12876, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
Evaluated 18.64% % of words ( 500 / 2683 )
[ Top 10 Memory Allocations ]
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\modules\linear.py:103: size=4947 KiB, count=63318, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:2486: size=2577 KiB, count=32978, average=80 B
<frozen importlib._bootstrap_external>:525: size=1364 KiB, count=8300, average=168 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:441: size=1237 KiB, count=15829, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:363: size=1237 KiB, count=15828, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\models\bert\modeling_bert.py:342: size=1237 KiB, count=15828, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\transformers\activations.py:56: size=1237 KiB, count=15828, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\torch\nn\functional.py:1818: size=1237 KiB, count=15828, average=80 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py:224: size=685 KiB, count=7197, average=98 B
P:\01_Informatik\01_Softwareprojekte\16_OntologyMapping\lib\site-packages\pandas\core\internals\managers.py:1957: size=356 KiB, count=2, average=178 KiB
Process finished with exit code 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment