prodigy.sh

$ prodigy dataset test

  ✨  Successfully added 'test' to database SQLite.

$ prodigy db-in test issues3.jsonl 

  ✨  Imported 1247 annotations for 'test' to database SQLite
  Added 'accept' answer to 1247 annotations
  Session ID: 2018-05-22_21-44-48

$ prodigy db-out test | head -1
{"text":"Podcast van 28 februari[...] vermissing, VVD","label":"wonen","meta":{"":"1","id":"188159157","medium":"1almere"},"_input_hash":-483749350,"_task_hash":-562927181,"answer":"accept"}

$ prodigy textcat.batch-train test -es 0.2 -n 1 --lang nl

Loaded blank model
Warning: Unnamed vectors -- this won't allow multiple vectors models to be loaded. (Shape: (0, 0))
Using 20% of examples (249) for evaluation
Using 100% of remaining examples (998) for training
Dropout: 0.2  Batch size: 10  Iterations: 1  

#          LOSS       F-SCORE    ACCURACY  
01         1319.093   0.994      0.988                                                                      

MODEL      USER       COUNT     
accept     accept     246       
accept     reject     0         
reject     reject     0         
reject     accept     3         

         
Correct    246
Incorrect  3

         
Baseline   1.00      
Precision  1.00      
Recall     0.99      
F-score    0.99      
Accuracy   0.99