Zhi Sun DOUDOU0314
DOUDOU0314
/ generation_example.py
Created
June 18, 2021 07:52
generation example of GPT-J-6B: 6B JAX-Based Transformer
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import time | |
| import torch | |
| from transformers import GPTNeoForCausalLM, AutoConfig, GPT2Tokenizer | |
| import torch | |
| import transformers | |
| import collections | |
| import os |
DOUDOU0314
/ comparison.txt
Last active
January 11, 2021 07:51
Comparison of deepspeed and gluon-nlp in SQuAD1.1
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Comparison of deepspeed and gluon-nlp in SQuAD1.1 | |
| Dataset:SQuAD1.1 | |
| GPUs:p3.8x with four teslaV100 | |
| BatchSize: 3 | |
| Epoch: 2 | |
| Max_seq_len: 384 | |
| Model: google_en_uncased_bert_wwm_large | |
| Results: | |
| deepspeed : 93.11/87.03 time cost: 0.71hours |