icemelon/bert_distilbert_cpu_inference.md

## bert_distilbert_cpu_inference.md

      
    Raw
  

              bert_distilbert_cpu_inference.md
            
          
model
batch
seq length
MXNet latency (ms)
TVM latency (ms)
speedup


BERT
1
64
26.1
12.6
2.1


BERT
1
128
45.8
19.2
2.4


BERT
1
256
99.4
35.3
2.8


DistilBERT
1
64
13.4
6.2
2.2


DistilBERT
1
128
23.2
9.5
2.5


DistilBERT
1
256
50.1
17.5
2.9
model	batch	seq length	MXNet latency (ms)	TVM latency (ms)	speedup
BERT	1	64	26.1	12.6	2.1
BERT	1	128	45.8	19.2	2.4
BERT	1	256	99.4	35.3	2.8
DistilBERT	1	64	13.4	6.2	2.2
DistilBERT	1	128	23.2	9.5	2.5
DistilBERT	1	256	50.1	17.5	2.9