Skip to content

Instantly share code, notes, and snippets.

@oserikov

oserikov/intents-classifier-demo.ipynb Secret

Created August 24, 2020 01:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save oserikov/b2ba7a0d5ea6a4b4624ca788e2ccbfd8 to your computer and use it in GitHub Desktop.
Save oserikov/b2ba7a0d5ea6a4b4624ca788e2ccbfd8 to your computer and use it in GitHub Desktop.
Download ZIP
intents classifier demo.ipynb
Display the source blob
Display the rendered blob
Raw
intents-classifier-demo.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "intents classifier demo.ipynb",
"provenance": [],
"toc_visible": true,
"authorship_tag": "ABX9TyPEAyHWCpENMI6S/dkiNMEk",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/oserikov/b2ba7a0d5ea6a4b4624ca788e2ccbfd8/intents-classifier-demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ir42jKLw_7-L",
"colab_type": "text"
},
"source": [
"# DeepPavlov Goal-Oriented Bot intents classification demo\n",
"\n",
"Below is a quick example of how train your own intents classificator to use in the go-bot pipeline."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GqOvhmAsAKNB",
"colab_type": "text"
},
"source": [
"### prerequisites installation"
]
},
{
"cell_type": "code",
"metadata": {
"id": "M0Lt0ufogkyb",
"colab_type": "code",
"colab": {}
},
"source": [
"%%capture install_cell_output\n",
"!pip install deeppavlov --progress-bar off\n",
"!python -m pip install tensorflow-gpu==1.15.2 --progress-bar off\n",
"!python -m deeppavlov install intents_dstc2\n",
"!python -m deeppavlov download intents_dstc2"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "psrdGFyCAZeI",
"colab_type": "text"
},
"source": [
"### imports"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yMmoGigFgxqp",
"colab_type": "code",
"colab": {}
},
"source": [
"from deeppavlov import build_model, train_model, configs\n",
"from deeppavlov.core.common.file import read_json"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "HPiJurgdAdmS",
"colab_type": "text"
},
"source": [
"### set-up the experiment config"
]
},
{
"cell_type": "code",
"metadata": {
"id": "-dZ8ojw_0Aig",
"colab_type": "code",
"colab": {}
},
"source": [
"intents_clf_config_di = read_json(configs.classifiers.intents_dstc2)"
],
"execution_count": 3,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "C2mWPE-L11l9",
"colab_type": "text"
},
"source": [
"Let's remove the pretrained model to train the new one from scratch."
]
},
{
"cell_type": "code",
"metadata": {
"id": "gE0hLG7lziqN",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 68
},
"outputId": "67b5fe35-72e7-420d-898a-5128678a0e29"
},
"source": [
"ROOT_PATH = intents_clf_config_di[\"metadata\"][\"variables\"][\"ROOT_PATH\"]\n",
"MODELS_PATH = intents_clf_config_di[\"metadata\"][\"variables\"][\"MODELS_PATH\"].format(ROOT_PATH=ROOT_PATH)\n",
"MODEL_PATH = intents_clf_config_di[\"metadata\"][\"variables\"][\"MODEL_PATH\"].format(MODELS_PATH=MODELS_PATH) \n",
"\n",
"print(\"ROOT_PATH: \" + ROOT_PATH, \"MODELS_PATH: \" + MODELS_PATH, \"MODEL_PATH: \" + MODEL_PATH, \n",
" sep='\\n')"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"ROOT_PATH: ~/.deeppavlov\n",
"MODELS_PATH: ~/.deeppavlov/models\n",
"MODEL_PATH: ~/.deeppavlov/models/classifiers/intents_dstc2_v10\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "cKctOCVP0CLD",
"colab_type": "code",
"colab": {}
},
"source": [
"!rm -r {MODEL_PATH}"
],
"execution_count": 5,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WBPL0Vh817ct",
"colab_type": "text"
},
"source": [
"Let's download the dstc2 data."
]
},
{
"cell_type": "code",
"metadata": {
"id": "t5jja5JgwdrG",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 204
},
"outputId": "d81b8c28-00a3-4d35-8445-8d65c3439bc7"
},
"source": [
"!wget http://files.deeppavlov.ai/datasets/dstc2_v3.tar.gz\n",
"!tar -zxf dstc2_v3.tar.gz"
],
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"text": [
"--2020-08-24 01:10:37-- http://files.deeppavlov.ai/datasets/dstc2_v3.tar.gz\n",
"Resolving files.deeppavlov.ai (files.deeppavlov.ai)... 93.175.29.74\n",
"Connecting to files.deeppavlov.ai (files.deeppavlov.ai)|93.175.29.74|:80... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 506452 (495K) [application/octet-stream]\n",
"Saving to: ‘dstc2_v3.tar.gz’\n",
"\n",
"dstc2_v3.tar.gz 100%[===================>] 494.58K 560KB/s in 0.9s \n",
"\n",
"2020-08-24 01:10:38 (560 KB/s) - ‘dstc2_v3.tar.gz’ saved [506452/506452]\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cYR0Emne40gl",
"colab_type": "text"
},
"source": [
"Let's take a quick look at user utterances and the respective intents.\n",
"In the dstc2 dataset, user replics have the `dialog_act` key that stores list of intents specified by user. Each intent name is stored under by the key `act`.\n",
"\n",
"See a short example below."
]
},
{
"cell_type": "code",
"metadata": {
"id": "GzBId4SH3XPx",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 187
},
"outputId": "9d7a13b8-1e63-4be0-85a4-ca3a97fb8db5"
},
"source": [
"!paste \\\n",
" <(grep \"speaker\\\": 1\" dstc2-trn.jsonlist | grep -o \"text\\\": \\\"[^\\\"]*\\\"\" | head) \\\n",
" <(grep \"speaker\\\": 1\" dstc2-trn.jsonlist \\\n",
" | grep -o \"dialog_acts.*\" \\\n",
" | sed \"s|dialog_acts\\\"\\:||g\" \\\n",
" | perl -pe \"s|\\\"slots\\\": \\[.*?\\], ||g\" \\\n",
" | head) \\\n",
" | column -t -s $'\\t'"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"text\": \"cheap restaurant\" [{\"act\": \"inform\"}]}\n",
"text\": \"any\" [{\"act\": \"inform\"}]}\n",
"text\": \"south\" [{\"act\": \"inform\"}]}\n",
"text\": \"address\" [{\"act\": \"request\"}]}\n",
"text\": \"phone number\" [{\"act\": \"request\"}]}\n",
"text\": \"thank you good bye\" [{\"act\": \"thankyou\"}, {\"act\": \"bye\"}]}\n",
"text\": \"gastropub\" [{\"act\": \"inform\"}]}\n",
"text\": \"gastropub\" [{\"act\": \"inform\"}]}\n",
"text\": \"gastropub food\" [{\"act\": \"inform\"}]}\n",
"text\": \"what is the address\" [{\"act\": \"request\"}]}\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tuh_QR6hA3dv",
"colab_type": "text"
},
"source": [
"### configure path to training dataset"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4fmACegb8O4X",
"colab_type": "text"
},
"source": [
"Since we extracted our dataset to the current folder, let's keep it here. We'll need to pass current directory as the `data_path`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "JabBfui1hofC",
"colab_type": "code",
"colab": {}
},
"source": [
"intents_clf_config_di[\"dataset_reader\"][\"data_path\"] = '.'"
],
"execution_count": 8,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "tw0bpZEyA9co",
"colab_type": "text"
},
"source": [
"### train the intents classification model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Uz68UFh18fY2",
"colab_type": "text"
},
"source": [
"Let's train our model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "7p45oTEAwkg9",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "bdb475b9-2112-4c3a-d816-f3d6ec97b372"
},
"source": [
"train_model(intents_clf_config_di)"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"2020-08-24 01:10:42.149 INFO in 'deeppavlov.dataset_readers.dstc2_reader'['dstc2_reader'] at line 112: [loading dialogs from /content/dstc2-trn.jsonlist]\n",
"2020-08-24 01:10:42.366 INFO in 'deeppavlov.dataset_readers.dstc2_reader'['dstc2_reader'] at line 112: [loading dialogs from /content/dstc2-val.jsonlist]\n",
"2020-08-24 01:10:42.537 INFO in 'deeppavlov.dataset_readers.dstc2_reader'['dstc2_reader'] at line 112: [loading dialogs from /content/dstc2-tst.jsonlist]\n",
"[nltk_data] Downloading package punkt to /root/nltk_data...\n",
"[nltk_data] Unzipping tokenizers/punkt.zip.\n",
"[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
"[nltk_data] Unzipping corpora/stopwords.zip.\n",
"[nltk_data] Downloading package perluniprops to /root/nltk_data...\n",
"[nltk_data] Unzipping misc/perluniprops.zip.\n",
"[nltk_data] Downloading package nonbreaking_prefixes to\n",
"[nltk_data] /root/nltk_data...\n",
"[nltk_data] Unzipping corpora/nonbreaking_prefixes.zip.\n",
"2020-08-24 01:10:44.388 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 101: [saving vocabulary to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/classes.dict]\n",
"2020-08-24 01:10:44.396 INFO in 'deeppavlov.models.embedders.fasttext_embedder'['fasttext_embedder'] at line 53: [loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/dstc2_fastText_model.bin`]\n",
"\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_backend.py:38: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:10:47.885 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 216: [initializing `KerasClassificationModel` from scratch as cnn_model]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"If using Keras pass *_constraint arguments to layers.\n",
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:10:48.357 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 129: Model was successfully initialized!\n",
"Model summary:\n",
"Model: \"model\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_1 (InputLayer) [(None, None, 100)] 0 \n",
"__________________________________________________________________________________________________\n",
"conv1d (Conv1D) (None, None, 512) 154112 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_1 (Conv1D) (None, None, 512) 256512 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_2 (Conv1D) (None, None, 512) 358912 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization (BatchNorma (None, None, 512) 2048 conv1d[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_1 (BatchNor (None, None, 512) 2048 conv1d_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_2 (BatchNor (None, None, 512) 2048 conv1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation (Activation) (None, None, 512) 0 batch_normalization[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_1 (Activation) (None, None, 512) 0 batch_normalization_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_2 (Activation) (None, None, 512) 0 batch_normalization_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d (GlobalMax (None, 512) 0 activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_1 (GlobalM (None, 512) 0 activation_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_2 (GlobalM (None, 512) 0 activation_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"concatenate (Concatenate) (None, 1536) 0 global_max_pooling1d[0][0] \n",
" global_max_pooling1d_1[0][0] \n",
" global_max_pooling1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout (Dropout) (None, 1536) 0 concatenate[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense (Dense) (None, 100) 153700 dropout[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_3 (BatchNor (None, 100) 400 dense[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_3 (Activation) (None, 100) 0 batch_normalization_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout_1 (Dropout) (None, 100) 0 activation_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_1 (Dense) (None, 28) 2828 dropout_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_4 (BatchNor (None, 28) 112 dense_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_4 (Activation) (None, 28) 0 batch_normalization_4[0][0] \n",
"==================================================================================================\n",
"Total params: 932,720\n",
"Trainable params: 929,392\n",
"Non-trainable params: 3,328\n",
"__________________________________________________________________________________________________\n",
"2020-08-24 01:10:57.42 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 199: Initial best sets_accuracy of 0.0\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.0, \"roc_auc\": 0.5685}, \"time_spent\": \"0:00:09\", \"epochs_done\": 0, \"batches_seen\": 0, \"train_examples_seen\": 0, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.0469, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:13\", \"epochs_done\": 0, \"batches_seen\": 100, \"train_examples_seen\": 6400, \"loss\": 0.5920355738699437}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.1406, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:14\", \"epochs_done\": 1, \"batches_seen\": 200, \"train_examples_seen\": 12756, \"loss\": 0.19562050074338913}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.0, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:15\", \"epochs_done\": 2, \"batches_seen\": 300, \"train_examples_seen\": 19112, \"loss\": 0.16977887958288193}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.0, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:16\", \"epochs_done\": 3, \"batches_seen\": 400, \"train_examples_seen\": 25468, \"loss\": 0.15619265541434288}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.0938, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:18\", \"epochs_done\": 3, \"batches_seen\": 500, \"train_examples_seen\": 31868, \"loss\": 0.14806892335414887}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.1094, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:19\", \"epochs_done\": 4, \"batches_seen\": 600, \"train_examples_seen\": 38224, \"loss\": 0.1418072009086609}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:07.808 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.1128\n",
"2020-08-24 01:11:07.809 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:07.811 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.1128, \"roc_auc\": 0.7964}, \"time_spent\": \"0:00:20\", \"epochs_done\": 5, \"batches_seen\": 640, \"train_examples_seen\": 40740, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.1406, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:21\", \"epochs_done\": 5, \"batches_seen\": 700, \"train_examples_seen\": 44580, \"loss\": 0.13767794959247112}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2031, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:22\", \"epochs_done\": 6, \"batches_seen\": 800, \"train_examples_seen\": 50936, \"loss\": 0.13464015170931817}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.1875, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:23\", \"epochs_done\": 7, \"batches_seen\": 900, \"train_examples_seen\": 57292, \"loss\": 0.12967348009347915}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.1719, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:24\", \"epochs_done\": 7, \"batches_seen\": 1000, \"train_examples_seen\": 63692, \"loss\": 0.12662222385406494}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2188, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:26\", \"epochs_done\": 8, \"batches_seen\": 1100, \"train_examples_seen\": 70048, \"loss\": 0.12406710028648377}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2344, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:27\", \"epochs_done\": 9, \"batches_seen\": 1200, \"train_examples_seen\": 76404, \"loss\": 0.12225220769643784}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:16.385 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.2316\n",
"2020-08-24 01:11:16.386 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:16.392 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.2316, \"roc_auc\": 0.8296}, \"time_spent\": \"0:00:29\", \"epochs_done\": 10, \"batches_seen\": 1280, \"train_examples_seen\": 81480, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3281, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:29\", \"epochs_done\": 10, \"batches_seen\": 1300, \"train_examples_seen\": 82760, \"loss\": 0.1199112693220377}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2969, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:30\", \"epochs_done\": 10, \"batches_seen\": 1400, \"train_examples_seen\": 89160, \"loss\": 0.11781261287629605}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2812, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:31\", \"epochs_done\": 11, \"batches_seen\": 1500, \"train_examples_seen\": 95516, \"loss\": 0.1153223691880703}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2969, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:32\", \"epochs_done\": 12, \"batches_seen\": 1600, \"train_examples_seen\": 101872, \"loss\": 0.11389404252171516}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2812, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:33\", \"epochs_done\": 13, \"batches_seen\": 1700, \"train_examples_seen\": 108228, \"loss\": 0.11246686063706875}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2969, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:35\", \"epochs_done\": 14, \"batches_seen\": 1800, \"train_examples_seen\": 114584, \"loss\": 0.1101948743313551}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2656, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:36\", \"epochs_done\": 14, \"batches_seen\": 1900, \"train_examples_seen\": 120984, \"loss\": 0.10826481319963932}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:24.502 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.262\n",
"2020-08-24 01:11:24.503 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:24.503 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.262, \"roc_auc\": 0.8609}, \"time_spent\": \"0:00:37\", \"epochs_done\": 15, \"batches_seen\": 1920, \"train_examples_seen\": 122220, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3125, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:38\", \"epochs_done\": 15, \"batches_seen\": 2000, \"train_examples_seen\": 127340, \"loss\": 0.10829331181943416}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3438, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:39\", \"epochs_done\": 16, \"batches_seen\": 2100, \"train_examples_seen\": 133696, \"loss\": 0.10704893633723259}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4688, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:40\", \"epochs_done\": 17, \"batches_seen\": 2200, \"train_examples_seen\": 140052, \"loss\": 0.10537146963179111}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3281, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:41\", \"epochs_done\": 17, \"batches_seen\": 2300, \"train_examples_seen\": 146452, \"loss\": 0.10408716149628162}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2656, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:42\", \"epochs_done\": 18, \"batches_seen\": 2400, \"train_examples_seen\": 152808, \"loss\": 0.10339649364352227}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3438, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:43\", \"epochs_done\": 19, \"batches_seen\": 2500, \"train_examples_seen\": 159164, \"loss\": 0.1026118667423725}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:32.687 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.2836\n",
"2020-08-24 01:11:32.688 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:32.692 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.2836, \"roc_auc\": 0.8593}, \"time_spent\": \"0:00:45\", \"epochs_done\": 20, \"batches_seen\": 2560, \"train_examples_seen\": 162960, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3594, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:45\", \"epochs_done\": 20, \"batches_seen\": 2600, \"train_examples_seen\": 165520, \"loss\": 0.10139974936842919}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2969, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:46\", \"epochs_done\": 21, \"batches_seen\": 2700, \"train_examples_seen\": 171876, \"loss\": 0.1009694817662239}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:48\", \"epochs_done\": 21, \"batches_seen\": 2800, \"train_examples_seen\": 178276, \"loss\": 0.09947541818022727}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4531, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:49\", \"epochs_done\": 22, \"batches_seen\": 2900, \"train_examples_seen\": 184632, \"loss\": 0.09942317388951778}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.375, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:50\", \"epochs_done\": 23, \"batches_seen\": 3000, \"train_examples_seen\": 190988, \"loss\": 0.09834368355572223}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3906, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:51\", \"epochs_done\": 24, \"batches_seen\": 3100, \"train_examples_seen\": 197344, \"loss\": 0.09842068389058113}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:52\", \"epochs_done\": 24, \"batches_seen\": 3200, \"train_examples_seen\": 203700, \"loss\": 0.09760622009634971}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:40.770 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.3366\n",
"2020-08-24 01:11:40.771 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:40.772 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.3366, \"roc_auc\": 0.8799}, \"time_spent\": \"0:00:53\", \"epochs_done\": 25, \"batches_seen\": 3200, \"train_examples_seen\": 203700, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4062, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:54\", \"epochs_done\": 25, \"batches_seen\": 3300, \"train_examples_seen\": 210100, \"loss\": 0.09738745607435703}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:55\", \"epochs_done\": 26, \"batches_seen\": 3400, \"train_examples_seen\": 216456, \"loss\": 0.09512191079556942}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.2656, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:56\", \"epochs_done\": 27, \"batches_seen\": 3500, \"train_examples_seen\": 222812, \"loss\": 0.09546162761747837}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:58\", \"epochs_done\": 28, \"batches_seen\": 3600, \"train_examples_seen\": 229168, \"loss\": 0.09500900894403458}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3438, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:59\", \"epochs_done\": 28, \"batches_seen\": 3700, \"train_examples_seen\": 235568, \"loss\": 0.09448700316250325}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4531, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:00\", \"epochs_done\": 29, \"batches_seen\": 3800, \"train_examples_seen\": 241924, \"loss\": 0.09292583651840687}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:48.816 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.3624\n",
"2020-08-24 01:11:48.817 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:48.818 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.3624, \"roc_auc\": 0.8927}, \"time_spent\": \"0:01:01\", \"epochs_done\": 30, \"batches_seen\": 3840, \"train_examples_seen\": 244440, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4219, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:02\", \"epochs_done\": 30, \"batches_seen\": 3900, \"train_examples_seen\": 248280, \"loss\": 0.09324058935046196}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4375, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:03\", \"epochs_done\": 31, \"batches_seen\": 4000, \"train_examples_seen\": 254636, \"loss\": 0.09364967375993728}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:04\", \"epochs_done\": 32, \"batches_seen\": 4100, \"train_examples_seen\": 260992, \"loss\": 0.09169360369443894}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:05\", \"epochs_done\": 32, \"batches_seen\": 4200, \"train_examples_seen\": 267392, \"loss\": 0.09195528484880924}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:06\", \"epochs_done\": 33, \"batches_seen\": 4300, \"train_examples_seen\": 273748, \"loss\": 0.0912359055876732}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:07\", \"epochs_done\": 34, \"batches_seen\": 4400, \"train_examples_seen\": 280104, \"loss\": 0.09000023707747459}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:11:56.922 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4284\n",
"2020-08-24 01:11:56.923 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:11:56.924 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4284, \"roc_auc\": 0.8777}, \"time_spent\": \"0:01:09\", \"epochs_done\": 35, \"batches_seen\": 4480, \"train_examples_seen\": 285180, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:09\", \"epochs_done\": 35, \"batches_seen\": 4500, \"train_examples_seen\": 286460, \"loss\": 0.09134131923317909}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:11\", \"epochs_done\": 35, \"batches_seen\": 4600, \"train_examples_seen\": 292860, \"loss\": 0.08978407055139542}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4688, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:12\", \"epochs_done\": 36, \"batches_seen\": 4700, \"train_examples_seen\": 299216, \"loss\": 0.0901230201870203}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4375, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:13\", \"epochs_done\": 37, \"batches_seen\": 4800, \"train_examples_seen\": 305572, \"loss\": 0.0903089129179716}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4688, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:14\", \"epochs_done\": 38, \"batches_seen\": 4900, \"train_examples_seen\": 311928, \"loss\": 0.08854238145053386}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4688, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:15\", \"epochs_done\": 39, \"batches_seen\": 5000, \"train_examples_seen\": 318284, \"loss\": 0.08851394705474376}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:16\", \"epochs_done\": 39, \"batches_seen\": 5100, \"train_examples_seen\": 324684, \"loss\": 0.08752539105713368}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:05.163 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4348\n",
"2020-08-24 01:12:05.164 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:05.165 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4348, \"roc_auc\": 0.897}, \"time_spent\": \"0:01:17\", \"epochs_done\": 40, \"batches_seen\": 5120, \"train_examples_seen\": 325920, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:18\", \"epochs_done\": 40, \"batches_seen\": 5200, \"train_examples_seen\": 331040, \"loss\": 0.08880603604018689}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4375, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:20\", \"epochs_done\": 41, \"batches_seen\": 5300, \"train_examples_seen\": 337396, \"loss\": 0.08740161292254925}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:21\", \"epochs_done\": 42, \"batches_seen\": 5400, \"train_examples_seen\": 343752, \"loss\": 0.08523834139108658}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5469, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:22\", \"epochs_done\": 42, \"batches_seen\": 5500, \"train_examples_seen\": 350152, \"loss\": 0.08722112521529198}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4375, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:23\", \"epochs_done\": 43, \"batches_seen\": 5600, \"train_examples_seen\": 356508, \"loss\": 0.08690966583788395}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.3906, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:24\", \"epochs_done\": 44, \"batches_seen\": 5700, \"train_examples_seen\": 362864, \"loss\": 0.08440400190651416}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:13.454 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4698\n",
"2020-08-24 01:12:13.455 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:13.457 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4698, \"roc_auc\": 0.8973}, \"time_spent\": \"0:01:26\", \"epochs_done\": 45, \"batches_seen\": 5760, \"train_examples_seen\": 366660, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:26\", \"epochs_done\": 45, \"batches_seen\": 5800, \"train_examples_seen\": 369220, \"loss\": 0.08534405343234538}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:27\", \"epochs_done\": 46, \"batches_seen\": 5900, \"train_examples_seen\": 375576, \"loss\": 0.08540759712457657}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4531, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:28\", \"epochs_done\": 46, \"batches_seen\": 6000, \"train_examples_seen\": 381976, \"loss\": 0.08483394227921963}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:30\", \"epochs_done\": 47, \"batches_seen\": 6100, \"train_examples_seen\": 388332, \"loss\": 0.08380240842700004}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:31\", \"epochs_done\": 48, \"batches_seen\": 6200, \"train_examples_seen\": 394688, \"loss\": 0.08509787566959857}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:32\", \"epochs_done\": 49, \"batches_seen\": 6300, \"train_examples_seen\": 401044, \"loss\": 0.08334489054977894}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4531, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:33\", \"epochs_done\": 49, \"batches_seen\": 6400, \"train_examples_seen\": 407400, \"loss\": 0.08223305270075798}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:21.719 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4724\n",
"2020-08-24 01:12:21.720 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:21.721 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4724, \"roc_auc\": 0.8984}, \"time_spent\": \"0:01:34\", \"epochs_done\": 50, \"batches_seen\": 6400, \"train_examples_seen\": 407400, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:35\", \"epochs_done\": 50, \"batches_seen\": 6500, \"train_examples_seen\": 413800, \"loss\": 0.08192475214600563}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:36\", \"epochs_done\": 51, \"batches_seen\": 6600, \"train_examples_seen\": 420156, \"loss\": 0.08272579796612263}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6094, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:37\", \"epochs_done\": 52, \"batches_seen\": 6700, \"train_examples_seen\": 426512, \"loss\": 0.08259044460952282}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:39\", \"epochs_done\": 53, \"batches_seen\": 6800, \"train_examples_seen\": 432868, \"loss\": 0.08154722016304732}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:40\", \"epochs_done\": 53, \"batches_seen\": 6900, \"train_examples_seen\": 439268, \"loss\": 0.08151682004332543}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:41\", \"epochs_done\": 54, \"batches_seen\": 7000, \"train_examples_seen\": 445624, \"loss\": 0.08189119938760996}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:29.916 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4862\n",
"2020-08-24 01:12:29.916 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:29.917 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4862, \"roc_auc\": 0.8874}, \"time_spent\": \"0:01:42\", \"epochs_done\": 55, \"batches_seen\": 7040, \"train_examples_seen\": 448140, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:43\", \"epochs_done\": 55, \"batches_seen\": 7100, \"train_examples_seen\": 451980, \"loss\": 0.08071204416453838}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:44\", \"epochs_done\": 56, \"batches_seen\": 7200, \"train_examples_seen\": 458336, \"loss\": 0.07935628280043602}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5469, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:45\", \"epochs_done\": 57, \"batches_seen\": 7300, \"train_examples_seen\": 464692, \"loss\": 0.08059127695858478}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:46\", \"epochs_done\": 57, \"batches_seen\": 7400, \"train_examples_seen\": 471092, \"loss\": 0.0803615079075098}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4531, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:47\", \"epochs_done\": 58, \"batches_seen\": 7500, \"train_examples_seen\": 477448, \"loss\": 0.07985691428184509}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:49\", \"epochs_done\": 59, \"batches_seen\": 7600, \"train_examples_seen\": 483804, \"loss\": 0.07989714138209819}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:38.46 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.4862\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4837, \"roc_auc\": 0.9048}, \"time_spent\": \"0:01:50\", \"epochs_done\": 60, \"batches_seen\": 7680, \"train_examples_seen\": 488880, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4062, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:50\", \"epochs_done\": 60, \"batches_seen\": 7700, \"train_examples_seen\": 490160, \"loss\": 0.07984212420880794}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:52\", \"epochs_done\": 60, \"batches_seen\": 7800, \"train_examples_seen\": 496560, \"loss\": 0.07884244576096534}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:53\", \"epochs_done\": 61, \"batches_seen\": 7900, \"train_examples_seen\": 502916, \"loss\": 0.07862173222005367}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:54\", \"epochs_done\": 62, \"batches_seen\": 8000, \"train_examples_seen\": 509272, \"loss\": 0.07964638441801071}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:55\", \"epochs_done\": 63, \"batches_seen\": 8100, \"train_examples_seen\": 515628, \"loss\": 0.0771608728915453}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:56\", \"epochs_done\": 64, \"batches_seen\": 8200, \"train_examples_seen\": 521984, \"loss\": 0.07794643986970186}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:57\", \"epochs_done\": 64, \"batches_seen\": 8300, \"train_examples_seen\": 528384, \"loss\": 0.07696234609931707}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:46.232 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4922\n",
"2020-08-24 01:12:46.233 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:46.235 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4922, \"roc_auc\": 0.907}, \"time_spent\": \"0:01:58\", \"epochs_done\": 65, \"batches_seen\": 8320, \"train_examples_seen\": 529620, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:01:59\", \"epochs_done\": 65, \"batches_seen\": 8400, \"train_examples_seen\": 534740, \"loss\": 0.07705731272697448}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:01\", \"epochs_done\": 66, \"batches_seen\": 8500, \"train_examples_seen\": 541096, \"loss\": 0.07787095487117768}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:02\", \"epochs_done\": 67, \"batches_seen\": 8600, \"train_examples_seen\": 547452, \"loss\": 0.07700474094599485}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:03\", \"epochs_done\": 67, \"batches_seen\": 8700, \"train_examples_seen\": 553852, \"loss\": 0.07667293466627598}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:04\", \"epochs_done\": 68, \"batches_seen\": 8800, \"train_examples_seen\": 560208, \"loss\": 0.07639208488166332}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:05\", \"epochs_done\": 69, \"batches_seen\": 8900, \"train_examples_seen\": 566564, \"loss\": 0.07657327126711606}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:12:54.424 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.4993\n",
"2020-08-24 01:12:54.425 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:12:54.426 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.4993, \"roc_auc\": 0.9127}, \"time_spent\": \"0:02:07\", \"epochs_done\": 70, \"batches_seen\": 8960, \"train_examples_seen\": 570360, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5469, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:07\", \"epochs_done\": 70, \"batches_seen\": 9000, \"train_examples_seen\": 572920, \"loss\": 0.07653588868677616}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:08\", \"epochs_done\": 71, \"batches_seen\": 9100, \"train_examples_seen\": 579276, \"loss\": 0.0756566235423088}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5938, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:09\", \"epochs_done\": 71, \"batches_seen\": 9200, \"train_examples_seen\": 585676, \"loss\": 0.07611226193606853}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:11\", \"epochs_done\": 72, \"batches_seen\": 9300, \"train_examples_seen\": 592032, \"loss\": 0.07530944541096687}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:12\", \"epochs_done\": 73, \"batches_seen\": 9400, \"train_examples_seen\": 598388, \"loss\": 0.07415907680988312}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:13\", \"epochs_done\": 74, \"batches_seen\": 9500, \"train_examples_seen\": 604744, \"loss\": 0.07396796133369207}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:14\", \"epochs_done\": 74, \"batches_seen\": 9600, \"train_examples_seen\": 611100, \"loss\": 0.07535308599472046}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:02.675 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.5244\n",
"2020-08-24 01:13:02.676 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:02.677 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.5244, \"roc_auc\": 0.9152}, \"time_spent\": \"0:02:15\", \"epochs_done\": 75, \"batches_seen\": 9600, \"train_examples_seen\": 611100, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:16\", \"epochs_done\": 75, \"batches_seen\": 9700, \"train_examples_seen\": 617500, \"loss\": 0.07367253050208092}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:17\", \"epochs_done\": 76, \"batches_seen\": 9800, \"train_examples_seen\": 623856, \"loss\": 0.07457991819828749}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:18\", \"epochs_done\": 77, \"batches_seen\": 9900, \"train_examples_seen\": 630212, \"loss\": 0.07440417345613241}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:20\", \"epochs_done\": 78, \"batches_seen\": 10000, \"train_examples_seen\": 636568, \"loss\": 0.07360821414738894}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:21\", \"epochs_done\": 78, \"batches_seen\": 10100, \"train_examples_seen\": 642968, \"loss\": 0.07348078742623329}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:22\", \"epochs_done\": 79, \"batches_seen\": 10200, \"train_examples_seen\": 649324, \"loss\": 0.07391441684216261}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:10.986 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.5414\n",
"2020-08-24 01:13:10.987 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:10.988 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.5414, \"roc_auc\": 0.9269}, \"time_spent\": \"0:02:23\", \"epochs_done\": 80, \"batches_seen\": 10240, \"train_examples_seen\": 651840, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:24\", \"epochs_done\": 80, \"batches_seen\": 10300, \"train_examples_seen\": 655680, \"loss\": 0.07305418498814106}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:25\", \"epochs_done\": 81, \"batches_seen\": 10400, \"train_examples_seen\": 662036, \"loss\": 0.07220228191465139}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6094, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:26\", \"epochs_done\": 82, \"batches_seen\": 10500, \"train_examples_seen\": 668392, \"loss\": 0.07354510642588138}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:27\", \"epochs_done\": 82, \"batches_seen\": 10600, \"train_examples_seen\": 674792, \"loss\": 0.07284882105886936}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:29\", \"epochs_done\": 83, \"batches_seen\": 10700, \"train_examples_seen\": 681148, \"loss\": 0.07189544830471277}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:30\", \"epochs_done\": 84, \"batches_seen\": 10800, \"train_examples_seen\": 687504, \"loss\": 0.07287140533328057}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:19.127 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.5624\n",
"2020-08-24 01:13:19.128 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:19.130 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.5624, \"roc_auc\": 0.9218}, \"time_spent\": \"0:02:31\", \"epochs_done\": 85, \"batches_seen\": 10880, \"train_examples_seen\": 692580, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:32\", \"epochs_done\": 85, \"batches_seen\": 10900, \"train_examples_seen\": 693860, \"loss\": 0.07177593365311623}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:33\", \"epochs_done\": 85, \"batches_seen\": 11000, \"train_examples_seen\": 700260, \"loss\": 0.07170565571635962}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.4844, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:34\", \"epochs_done\": 86, \"batches_seen\": 11100, \"train_examples_seen\": 706616, \"loss\": 0.07143888156861067}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:35\", \"epochs_done\": 87, \"batches_seen\": 11200, \"train_examples_seen\": 712972, \"loss\": 0.071394312530756}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6094, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:36\", \"epochs_done\": 88, \"batches_seen\": 11300, \"train_examples_seen\": 719328, \"loss\": 0.07168154645711183}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:37\", \"epochs_done\": 89, \"batches_seen\": 11400, \"train_examples_seen\": 725684, \"loss\": 0.07043683160096408}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:39\", \"epochs_done\": 89, \"batches_seen\": 11500, \"train_examples_seen\": 732084, \"loss\": 0.0708996882662177}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:27.302 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.5932\n",
"2020-08-24 01:13:27.303 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:27.304 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.5932, \"roc_auc\": 0.9263}, \"time_spent\": \"0:02:39\", \"epochs_done\": 90, \"batches_seen\": 11520, \"train_examples_seen\": 733320, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5312, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:40\", \"epochs_done\": 90, \"batches_seen\": 11600, \"train_examples_seen\": 738440, \"loss\": 0.07090672753751277}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:42\", \"epochs_done\": 91, \"batches_seen\": 11700, \"train_examples_seen\": 744796, \"loss\": 0.06933800280094146}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:43\", \"epochs_done\": 92, \"batches_seen\": 11800, \"train_examples_seen\": 751152, \"loss\": 0.07019073951989413}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:44\", \"epochs_done\": 92, \"batches_seen\": 11900, \"train_examples_seen\": 757552, \"loss\": 0.06942272085696459}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:45\", \"epochs_done\": 93, \"batches_seen\": 12000, \"train_examples_seen\": 763908, \"loss\": 0.07027892220765353}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:46\", \"epochs_done\": 94, \"batches_seen\": 12100, \"train_examples_seen\": 770264, \"loss\": 0.06935665886849166}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:35.525 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.5957\n",
"2020-08-24 01:13:35.526 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:35.528 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.5957, \"roc_auc\": 0.9284}, \"time_spent\": \"0:02:48\", \"epochs_done\": 95, \"batches_seen\": 12160, \"train_examples_seen\": 774060, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:48\", \"epochs_done\": 95, \"batches_seen\": 12200, \"train_examples_seen\": 776620, \"loss\": 0.06882044155150652}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:49\", \"epochs_done\": 96, \"batches_seen\": 12300, \"train_examples_seen\": 782976, \"loss\": 0.06995773900300264}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6094, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:51\", \"epochs_done\": 96, \"batches_seen\": 12400, \"train_examples_seen\": 789376, \"loss\": 0.06917541943490506}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:52\", \"epochs_done\": 97, \"batches_seen\": 12500, \"train_examples_seen\": 795732, \"loss\": 0.06963057160377502}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:53\", \"epochs_done\": 98, \"batches_seen\": 12600, \"train_examples_seen\": 802088, \"loss\": 0.06788570515811443}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:54\", \"epochs_done\": 99, \"batches_seen\": 12700, \"train_examples_seen\": 808444, \"loss\": 0.07028026323765517}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:55\", \"epochs_done\": 99, \"batches_seen\": 12800, \"train_examples_seen\": 814800, \"loss\": 0.06800760317593812}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:43.685 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6056\n",
"2020-08-24 01:13:43.686 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:43.687 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6056, \"roc_auc\": 0.931}, \"time_spent\": \"0:02:56\", \"epochs_done\": 100, \"batches_seen\": 12800, \"train_examples_seen\": 814800, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:57\", \"epochs_done\": 100, \"batches_seen\": 12900, \"train_examples_seen\": 821200, \"loss\": 0.06822371739894152}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:58\", \"epochs_done\": 101, \"batches_seen\": 13000, \"train_examples_seen\": 827556, \"loss\": 0.0685129565745592}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:02:59\", \"epochs_done\": 102, \"batches_seen\": 13100, \"train_examples_seen\": 833912, \"loss\": 0.06815227627754211}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:00\", \"epochs_done\": 103, \"batches_seen\": 13200, \"train_examples_seen\": 840268, \"loss\": 0.06790227040648461}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:02\", \"epochs_done\": 103, \"batches_seen\": 13300, \"train_examples_seen\": 846668, \"loss\": 0.06841212499886751}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5156, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:03\", \"epochs_done\": 104, \"batches_seen\": 13400, \"train_examples_seen\": 853024, \"loss\": 0.06764348108321429}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:51.760 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6059\n",
"2020-08-24 01:13:51.761 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:51.762 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6059, \"roc_auc\": 0.931}, \"time_spent\": \"0:03:04\", \"epochs_done\": 105, \"batches_seen\": 13440, \"train_examples_seen\": 855540, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:05\", \"epochs_done\": 105, \"batches_seen\": 13500, \"train_examples_seen\": 859380, \"loss\": 0.06804556392133236}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:06\", \"epochs_done\": 106, \"batches_seen\": 13600, \"train_examples_seen\": 865736, \"loss\": 0.06747688438743353}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:07\", \"epochs_done\": 107, \"batches_seen\": 13700, \"train_examples_seen\": 872092, \"loss\": 0.06774207055568696}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5625, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:08\", \"epochs_done\": 107, \"batches_seen\": 13800, \"train_examples_seen\": 878492, \"loss\": 0.06695581287145615}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:09\", \"epochs_done\": 108, \"batches_seen\": 13900, \"train_examples_seen\": 884848, \"loss\": 0.0670920767262578}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:10\", \"epochs_done\": 109, \"batches_seen\": 14000, \"train_examples_seen\": 891204, \"loss\": 0.06705922577530146}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:13:59.906 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6225\n",
"2020-08-24 01:13:59.907 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:13:59.908 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6225, \"roc_auc\": 0.9358}, \"time_spent\": \"0:03:12\", \"epochs_done\": 110, \"batches_seen\": 14080, \"train_examples_seen\": 896280, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:12\", \"epochs_done\": 110, \"batches_seen\": 14100, \"train_examples_seen\": 897560, \"loss\": 0.06518506042659283}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:14\", \"epochs_done\": 110, \"batches_seen\": 14200, \"train_examples_seen\": 903960, \"loss\": 0.06642080549150706}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:15\", \"epochs_done\": 111, \"batches_seen\": 14300, \"train_examples_seen\": 910316, \"loss\": 0.06652185473591089}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:16\", \"epochs_done\": 112, \"batches_seen\": 14400, \"train_examples_seen\": 916672, \"loss\": 0.06693271361291409}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:17\", \"epochs_done\": 113, \"batches_seen\": 14500, \"train_examples_seen\": 923028, \"loss\": 0.06552791941910982}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5938, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:18\", \"epochs_done\": 114, \"batches_seen\": 14600, \"train_examples_seen\": 929384, \"loss\": 0.06634157877415418}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:19\", \"epochs_done\": 114, \"batches_seen\": 14700, \"train_examples_seen\": 935784, \"loss\": 0.06553895384073258}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:08.41 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6225\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6208, \"roc_auc\": 0.9325}, \"time_spent\": \"0:03:20\", \"epochs_done\": 115, \"batches_seen\": 14720, \"train_examples_seen\": 937020, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:21\", \"epochs_done\": 115, \"batches_seen\": 14800, \"train_examples_seen\": 942140, \"loss\": 0.06590495686978101}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:22\", \"epochs_done\": 116, \"batches_seen\": 14900, \"train_examples_seen\": 948496, \"loss\": 0.06517591468989849}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:23\", \"epochs_done\": 117, \"batches_seen\": 15000, \"train_examples_seen\": 954852, \"loss\": 0.065744404271245}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:25\", \"epochs_done\": 117, \"batches_seen\": 15100, \"train_examples_seen\": 961252, \"loss\": 0.06451422482728958}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:26\", \"epochs_done\": 118, \"batches_seen\": 15200, \"train_examples_seen\": 967608, \"loss\": 0.06445816043764353}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:27\", \"epochs_done\": 119, \"batches_seen\": 15300, \"train_examples_seen\": 973964, \"loss\": 0.06662874061614275}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:16.68 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.63\n",
"2020-08-24 01:14:16.69 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:14:16.70 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.63, \"roc_auc\": 0.9342}, \"time_spent\": \"0:03:28\", \"epochs_done\": 120, \"batches_seen\": 15360, \"train_examples_seen\": 977760, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:29\", \"epochs_done\": 120, \"batches_seen\": 15400, \"train_examples_seen\": 980320, \"loss\": 0.06469743844121695}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:30\", \"epochs_done\": 121, \"batches_seen\": 15500, \"train_examples_seen\": 986676, \"loss\": 0.06478434298187494}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:31\", \"epochs_done\": 121, \"batches_seen\": 15600, \"train_examples_seen\": 993076, \"loss\": 0.0645980902761221}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:32\", \"epochs_done\": 122, \"batches_seen\": 15700, \"train_examples_seen\": 999432, \"loss\": 0.06539189599454404}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:33\", \"epochs_done\": 123, \"batches_seen\": 15800, \"train_examples_seen\": 1005788, \"loss\": 0.06450121749192476}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:35\", \"epochs_done\": 124, \"batches_seen\": 15900, \"train_examples_seen\": 1012144, \"loss\": 0.06416046429425477}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:36\", \"epochs_done\": 124, \"batches_seen\": 16000, \"train_examples_seen\": 1018500, \"loss\": 0.06465603306889534}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:24.251 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.63\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6218, \"roc_auc\": 0.9368}, \"time_spent\": \"0:03:36\", \"epochs_done\": 125, \"batches_seen\": 16000, \"train_examples_seen\": 1018500, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:38\", \"epochs_done\": 125, \"batches_seen\": 16100, \"train_examples_seen\": 1024900, \"loss\": 0.06476470421999693}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:39\", \"epochs_done\": 126, \"batches_seen\": 16200, \"train_examples_seen\": 1031256, \"loss\": 0.06384975086897611}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:40\", \"epochs_done\": 127, \"batches_seen\": 16300, \"train_examples_seen\": 1037612, \"loss\": 0.06294804155826568}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:41\", \"epochs_done\": 128, \"batches_seen\": 16400, \"train_examples_seen\": 1043968, \"loss\": 0.06401487685739994}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:42\", \"epochs_done\": 128, \"batches_seen\": 16500, \"train_examples_seen\": 1050368, \"loss\": 0.06310596290975809}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:43\", \"epochs_done\": 129, \"batches_seen\": 16600, \"train_examples_seen\": 1056724, \"loss\": 0.06459986846894025}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:32.500 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6365\n",
"2020-08-24 01:14:32.501 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:14:32.503 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6365, \"roc_auc\": 0.9416}, \"time_spent\": \"0:03:45\", \"epochs_done\": 130, \"batches_seen\": 16640, \"train_examples_seen\": 1059240, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:45\", \"epochs_done\": 130, \"batches_seen\": 16700, \"train_examples_seen\": 1063080, \"loss\": 0.06374722555279731}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5938, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:47\", \"epochs_done\": 131, \"batches_seen\": 16800, \"train_examples_seen\": 1069436, \"loss\": 0.06252010643482209}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:48\", \"epochs_done\": 132, \"batches_seen\": 16900, \"train_examples_seen\": 1075792, \"loss\": 0.0641694912314415}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:49\", \"epochs_done\": 132, \"batches_seen\": 17000, \"train_examples_seen\": 1082192, \"loss\": 0.06308602049946785}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5781, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:50\", \"epochs_done\": 133, \"batches_seen\": 17100, \"train_examples_seen\": 1088548, \"loss\": 0.06216529484838247}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:51\", \"epochs_done\": 134, \"batches_seen\": 17200, \"train_examples_seen\": 1094904, \"loss\": 0.062352015897631646}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:40.715 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6604\n",
"2020-08-24 01:14:40.715 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:14:40.716 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6604, \"roc_auc\": 0.9401}, \"time_spent\": \"0:03:53\", \"epochs_done\": 135, \"batches_seen\": 17280, \"train_examples_seen\": 1099980, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:53\", \"epochs_done\": 135, \"batches_seen\": 17300, \"train_examples_seen\": 1101260, \"loss\": 0.06290873598307371}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:54\", \"epochs_done\": 135, \"batches_seen\": 17400, \"train_examples_seen\": 1107660, \"loss\": 0.06217392053455115}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:55\", \"epochs_done\": 136, \"batches_seen\": 17500, \"train_examples_seen\": 1114016, \"loss\": 0.06246035009622574}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:57\", \"epochs_done\": 137, \"batches_seen\": 17600, \"train_examples_seen\": 1120372, \"loss\": 0.0627869813144207}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5938, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:58\", \"epochs_done\": 138, \"batches_seen\": 17700, \"train_examples_seen\": 1126728, \"loss\": 0.06284515552222729}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:03:59\", \"epochs_done\": 139, \"batches_seen\": 17800, \"train_examples_seen\": 1133084, \"loss\": 0.06195865243673324}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:00\", \"epochs_done\": 139, \"batches_seen\": 17900, \"train_examples_seen\": 1139484, \"loss\": 0.061447510421276094}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:48.872 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6604\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6552, \"roc_auc\": 0.9421}, \"time_spent\": \"0:04:01\", \"epochs_done\": 140, \"batches_seen\": 17920, \"train_examples_seen\": 1140720, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:02\", \"epochs_done\": 140, \"batches_seen\": 18000, \"train_examples_seen\": 1145840, \"loss\": 0.06250710062682628}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:03\", \"epochs_done\": 141, \"batches_seen\": 18100, \"train_examples_seen\": 1152196, \"loss\": 0.060948454812169074}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:04\", \"epochs_done\": 142, \"batches_seen\": 18200, \"train_examples_seen\": 1158552, \"loss\": 0.06162753455340862}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:05\", \"epochs_done\": 142, \"batches_seen\": 18300, \"train_examples_seen\": 1164952, \"loss\": 0.0611849033460021}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:07\", \"epochs_done\": 143, \"batches_seen\": 18400, \"train_examples_seen\": 1171308, \"loss\": 0.061653579473495486}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:08\", \"epochs_done\": 144, \"batches_seen\": 18500, \"train_examples_seen\": 1177664, \"loss\": 0.061674288101494314}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:14:56.950 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6604\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6549, \"roc_auc\": 0.9407}, \"time_spent\": \"0:04:09\", \"epochs_done\": 145, \"batches_seen\": 18560, \"train_examples_seen\": 1181460, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:10\", \"epochs_done\": 145, \"batches_seen\": 18600, \"train_examples_seen\": 1184020, \"loss\": 0.062228729017078876}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:11\", \"epochs_done\": 146, \"batches_seen\": 18700, \"train_examples_seen\": 1190376, \"loss\": 0.06148001674562693}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:12\", \"epochs_done\": 146, \"batches_seen\": 18800, \"train_examples_seen\": 1196776, \"loss\": 0.060450673848390576}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:13\", \"epochs_done\": 147, \"batches_seen\": 18900, \"train_examples_seen\": 1203132, \"loss\": 0.061055817268788816}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:14\", \"epochs_done\": 148, \"batches_seen\": 19000, \"train_examples_seen\": 1209488, \"loss\": 0.06106316115707159}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:15\", \"epochs_done\": 149, \"batches_seen\": 19100, \"train_examples_seen\": 1215844, \"loss\": 0.06138202223926783}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:16\", \"epochs_done\": 149, \"batches_seen\": 19200, \"train_examples_seen\": 1222200, \"loss\": 0.059916167855262756}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:05.31 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.665\n",
"2020-08-24 01:15:05.32 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:15:05.33 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.665, \"roc_auc\": 0.9429}, \"time_spent\": \"0:04:17\", \"epochs_done\": 150, \"batches_seen\": 19200, \"train_examples_seen\": 1222200, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:18\", \"epochs_done\": 150, \"batches_seen\": 19300, \"train_examples_seen\": 1228600, \"loss\": 0.0605414055660367}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.625, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:19\", \"epochs_done\": 151, \"batches_seen\": 19400, \"train_examples_seen\": 1234956, \"loss\": 0.06112995367497206}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:21\", \"epochs_done\": 152, \"batches_seen\": 19500, \"train_examples_seen\": 1241312, \"loss\": 0.06056448839604855}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:22\", \"epochs_done\": 153, \"batches_seen\": 19600, \"train_examples_seen\": 1247668, \"loss\": 0.060870607011020185}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:23\", \"epochs_done\": 153, \"batches_seen\": 19700, \"train_examples_seen\": 1254068, \"loss\": 0.0598438761010766}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:24\", \"epochs_done\": 154, \"batches_seen\": 19800, \"train_examples_seen\": 1260424, \"loss\": 0.06037930179387331}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:13.99 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.665\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6621, \"roc_auc\": 0.9461}, \"time_spent\": \"0:04:25\", \"epochs_done\": 155, \"batches_seen\": 19840, \"train_examples_seen\": 1262940, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:26\", \"epochs_done\": 155, \"batches_seen\": 19900, \"train_examples_seen\": 1266780, \"loss\": 0.060385794900357725}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:27\", \"epochs_done\": 156, \"batches_seen\": 20000, \"train_examples_seen\": 1273136, \"loss\": 0.05945237550884485}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:28\", \"epochs_done\": 157, \"batches_seen\": 20100, \"train_examples_seen\": 1279492, \"loss\": 0.06040815960615873}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:29\", \"epochs_done\": 157, \"batches_seen\": 20200, \"train_examples_seen\": 1285892, \"loss\": 0.06023860514163971}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:31\", \"epochs_done\": 158, \"batches_seen\": 20300, \"train_examples_seen\": 1292248, \"loss\": 0.05878803871572018}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:32\", \"epochs_done\": 159, \"batches_seen\": 20400, \"train_examples_seen\": 1298604, \"loss\": 0.06064199585467577}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:21.120 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.665\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6646, \"roc_auc\": 0.9453}, \"time_spent\": \"0:04:33\", \"epochs_done\": 160, \"batches_seen\": 20480, \"train_examples_seen\": 1303680, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:34\", \"epochs_done\": 160, \"batches_seen\": 20500, \"train_examples_seen\": 1304960, \"loss\": 0.05888104390352964}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:35\", \"epochs_done\": 160, \"batches_seen\": 20600, \"train_examples_seen\": 1311360, \"loss\": 0.05932281423360109}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:36\", \"epochs_done\": 161, \"batches_seen\": 20700, \"train_examples_seen\": 1317716, \"loss\": 0.05980074118822813}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:37\", \"epochs_done\": 162, \"batches_seen\": 20800, \"train_examples_seen\": 1324072, \"loss\": 0.0591557203233242}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:38\", \"epochs_done\": 163, \"batches_seen\": 20900, \"train_examples_seen\": 1330428, \"loss\": 0.058712230212986466}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:39\", \"epochs_done\": 164, \"batches_seen\": 21000, \"train_examples_seen\": 1336784, \"loss\": 0.0588831077888608}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:40\", \"epochs_done\": 164, \"batches_seen\": 21100, \"train_examples_seen\": 1343184, \"loss\": 0.05974790424108505}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:29.138 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.665\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6561, \"roc_auc\": 0.9456}, \"time_spent\": \"0:04:41\", \"epochs_done\": 165, \"batches_seen\": 21120, \"train_examples_seen\": 1344420, \"impatience\": 3, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:42\", \"epochs_done\": 165, \"batches_seen\": 21200, \"train_examples_seen\": 1349540, \"loss\": 0.05883199989795685}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:43\", \"epochs_done\": 166, \"batches_seen\": 21300, \"train_examples_seen\": 1355896, \"loss\": 0.0588994200900197}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:45\", \"epochs_done\": 167, \"batches_seen\": 21400, \"train_examples_seen\": 1362252, \"loss\": 0.059287572018802165}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:46\", \"epochs_done\": 167, \"batches_seen\": 21500, \"train_examples_seen\": 1368652, \"loss\": 0.058159238286316396}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:47\", \"epochs_done\": 168, \"batches_seen\": 21600, \"train_examples_seen\": 1375008, \"loss\": 0.059670842587947845}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:48\", \"epochs_done\": 169, \"batches_seen\": 21700, \"train_examples_seen\": 1381364, \"loss\": 0.056773792281746865}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:37.242 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.665\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6582, \"roc_auc\": 0.9438}, \"time_spent\": \"0:04:49\", \"epochs_done\": 170, \"batches_seen\": 21760, \"train_examples_seen\": 1385160, \"impatience\": 4, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:50\", \"epochs_done\": 170, \"batches_seen\": 21800, \"train_examples_seen\": 1387720, \"loss\": 0.05855325326323509}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:51\", \"epochs_done\": 171, \"batches_seen\": 21900, \"train_examples_seen\": 1394076, \"loss\": 0.05788210790604353}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6406, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:52\", \"epochs_done\": 171, \"batches_seen\": 22000, \"train_examples_seen\": 1400476, \"loss\": 0.0589709273353219}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:53\", \"epochs_done\": 172, \"batches_seen\": 22100, \"train_examples_seen\": 1406832, \"loss\": 0.057518749609589574}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:55\", \"epochs_done\": 173, \"batches_seen\": 22200, \"train_examples_seen\": 1413188, \"loss\": 0.05841941952705383}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:56\", \"epochs_done\": 174, \"batches_seen\": 22300, \"train_examples_seen\": 1419544, \"loss\": 0.057553206235170365}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:57\", \"epochs_done\": 174, \"batches_seen\": 22400, \"train_examples_seen\": 1425900, \"loss\": 0.05791358295828104}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:45.357 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6764\n",
"2020-08-24 01:15:45.358 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:15:45.358 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6764, \"roc_auc\": 0.9471}, \"time_spent\": \"0:04:57\", \"epochs_done\": 175, \"batches_seen\": 22400, \"train_examples_seen\": 1425900, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:04:59\", \"epochs_done\": 175, \"batches_seen\": 22500, \"train_examples_seen\": 1432300, \"loss\": 0.05724990658462047}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:00\", \"epochs_done\": 176, \"batches_seen\": 22600, \"train_examples_seen\": 1438656, \"loss\": 0.058877068869769573}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:01\", \"epochs_done\": 177, \"batches_seen\": 22700, \"train_examples_seen\": 1445012, \"loss\": 0.056741040907800196}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:02\", \"epochs_done\": 178, \"batches_seen\": 22800, \"train_examples_seen\": 1451368, \"loss\": 0.05773836974054575}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:03\", \"epochs_done\": 178, \"batches_seen\": 22900, \"train_examples_seen\": 1457768, \"loss\": 0.05807469382882118}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:04\", \"epochs_done\": 179, \"batches_seen\": 23000, \"train_examples_seen\": 1464124, \"loss\": 0.05739394258707762}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:15:53.438 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6764\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6731, \"roc_auc\": 0.9461}, \"time_spent\": \"0:05:06\", \"epochs_done\": 180, \"batches_seen\": 23040, \"train_examples_seen\": 1466640, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:06\", \"epochs_done\": 180, \"batches_seen\": 23100, \"train_examples_seen\": 1470480, \"loss\": 0.05825916185975075}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:07\", \"epochs_done\": 181, \"batches_seen\": 23200, \"train_examples_seen\": 1476836, \"loss\": 0.05671818468719721}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:09\", \"epochs_done\": 182, \"batches_seen\": 23300, \"train_examples_seen\": 1483192, \"loss\": 0.05728580866008997}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:10\", \"epochs_done\": 182, \"batches_seen\": 23400, \"train_examples_seen\": 1489592, \"loss\": 0.057732279151678084}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:11\", \"epochs_done\": 183, \"batches_seen\": 23500, \"train_examples_seen\": 1495948, \"loss\": 0.05747765384614468}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:12\", \"epochs_done\": 184, \"batches_seen\": 23600, \"train_examples_seen\": 1502304, \"loss\": 0.05612827152013779}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:01.476 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6766\n",
"2020-08-24 01:16:01.477 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:16:01.478 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6766, \"roc_auc\": 0.9442}, \"time_spent\": \"0:05:14\", \"epochs_done\": 185, \"batches_seen\": 23680, \"train_examples_seen\": 1507380, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:14\", \"epochs_done\": 185, \"batches_seen\": 23700, \"train_examples_seen\": 1508660, \"loss\": 0.0566171682626009}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:15\", \"epochs_done\": 185, \"batches_seen\": 23800, \"train_examples_seen\": 1515060, \"loss\": 0.05692241791635752}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:16\", \"epochs_done\": 186, \"batches_seen\": 23900, \"train_examples_seen\": 1521416, \"loss\": 0.056633607521653175}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:17\", \"epochs_done\": 187, \"batches_seen\": 24000, \"train_examples_seen\": 1527772, \"loss\": 0.0561417181417346}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:19\", \"epochs_done\": 188, \"batches_seen\": 24100, \"train_examples_seen\": 1534128, \"loss\": 0.057670221142470834}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:20\", \"epochs_done\": 189, \"batches_seen\": 24200, \"train_examples_seen\": 1540484, \"loss\": 0.056479194574058054}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.5938, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:21\", \"epochs_done\": 189, \"batches_seen\": 24300, \"train_examples_seen\": 1546884, \"loss\": 0.056373291015625}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:09.601 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6766\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6757, \"roc_auc\": 0.9417}, \"time_spent\": \"0:05:22\", \"epochs_done\": 190, \"batches_seen\": 24320, \"train_examples_seen\": 1548120, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:23\", \"epochs_done\": 190, \"batches_seen\": 24400, \"train_examples_seen\": 1553240, \"loss\": 0.057479820288717745}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:24\", \"epochs_done\": 191, \"batches_seen\": 24500, \"train_examples_seen\": 1559596, \"loss\": 0.05574306514114141}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:25\", \"epochs_done\": 192, \"batches_seen\": 24600, \"train_examples_seen\": 1565952, \"loss\": 0.05726686991751194}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:26\", \"epochs_done\": 192, \"batches_seen\": 24700, \"train_examples_seen\": 1572352, \"loss\": 0.0556796920672059}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:27\", \"epochs_done\": 193, \"batches_seen\": 24800, \"train_examples_seen\": 1578708, \"loss\": 0.056888962537050246}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:28\", \"epochs_done\": 194, \"batches_seen\": 24900, \"train_examples_seen\": 1585064, \"loss\": 0.05638667296618223}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:17.685 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6766\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6685, \"roc_auc\": 0.9421}, \"time_spent\": \"0:05:30\", \"epochs_done\": 195, \"batches_seen\": 24960, \"train_examples_seen\": 1588860, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:30\", \"epochs_done\": 195, \"batches_seen\": 25000, \"train_examples_seen\": 1591420, \"loss\": 0.05567652933299541}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:31\", \"epochs_done\": 196, \"batches_seen\": 25100, \"train_examples_seen\": 1597776, \"loss\": 0.05561559870839119}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:33\", \"epochs_done\": 196, \"batches_seen\": 25200, \"train_examples_seen\": 1604176, \"loss\": 0.05700810451060533}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:34\", \"epochs_done\": 197, \"batches_seen\": 25300, \"train_examples_seen\": 1610532, \"loss\": 0.05550652101635933}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:35\", \"epochs_done\": 198, \"batches_seen\": 25400, \"train_examples_seen\": 1616888, \"loss\": 0.055797508843243124}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:36\", \"epochs_done\": 199, \"batches_seen\": 25500, \"train_examples_seen\": 1623244, \"loss\": 0.055798764266073705}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6562, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:37\", \"epochs_done\": 199, \"batches_seen\": 25600, \"train_examples_seen\": 1629600, \"loss\": 0.055227341651916506}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:25.743 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6768\n",
"2020-08-24 01:16:25.744 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:16:25.745 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6768, \"roc_auc\": 0.9443}, \"time_spent\": \"0:05:38\", \"epochs_done\": 200, \"batches_seen\": 25600, \"train_examples_seen\": 1629600, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:39\", \"epochs_done\": 200, \"batches_seen\": 25700, \"train_examples_seen\": 1636000, \"loss\": 0.05537706173956394}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:40\", \"epochs_done\": 201, \"batches_seen\": 25800, \"train_examples_seen\": 1642356, \"loss\": 0.0554152974113822}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:41\", \"epochs_done\": 202, \"batches_seen\": 25900, \"train_examples_seen\": 1648712, \"loss\": 0.055948162749409676}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:43\", \"epochs_done\": 203, \"batches_seen\": 26000, \"train_examples_seen\": 1655068, \"loss\": 0.05629071194678545}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:44\", \"epochs_done\": 203, \"batches_seen\": 26100, \"train_examples_seen\": 1661468, \"loss\": 0.05483669832348823}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:45\", \"epochs_done\": 204, \"batches_seen\": 26200, \"train_examples_seen\": 1667824, \"loss\": 0.05513660050928593}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:33.869 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.6934\n",
"2020-08-24 01:16:33.870 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:16:33.871 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6934, \"roc_auc\": 0.9441}, \"time_spent\": \"0:05:46\", \"epochs_done\": 205, \"batches_seen\": 26240, \"train_examples_seen\": 1670340, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:47\", \"epochs_done\": 205, \"batches_seen\": 26300, \"train_examples_seen\": 1674180, \"loss\": 0.05526880130171776}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:48\", \"epochs_done\": 206, \"batches_seen\": 26400, \"train_examples_seen\": 1680536, \"loss\": 0.0550919909030199}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:49\", \"epochs_done\": 207, \"batches_seen\": 26500, \"train_examples_seen\": 1686892, \"loss\": 0.05512629486620426}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:50\", \"epochs_done\": 207, \"batches_seen\": 26600, \"train_examples_seen\": 1693292, \"loss\": 0.05502007804811001}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:51\", \"epochs_done\": 208, \"batches_seen\": 26700, \"train_examples_seen\": 1699648, \"loss\": 0.05529864724725485}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:53\", \"epochs_done\": 209, \"batches_seen\": 26800, \"train_examples_seen\": 1706004, \"loss\": 0.054636777341365815}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:42.166 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6934\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6922, \"roc_auc\": 0.9447}, \"time_spent\": \"0:05:54\", \"epochs_done\": 210, \"batches_seen\": 26880, \"train_examples_seen\": 1711080, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:55\", \"epochs_done\": 210, \"batches_seen\": 26900, \"train_examples_seen\": 1712360, \"loss\": 0.05444038707762957}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:56\", \"epochs_done\": 210, \"batches_seen\": 27000, \"train_examples_seen\": 1718760, \"loss\": 0.05476227380335331}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:57\", \"epochs_done\": 211, \"batches_seen\": 27100, \"train_examples_seen\": 1725116, \"loss\": 0.054959292635321615}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:58\", \"epochs_done\": 212, \"batches_seen\": 27200, \"train_examples_seen\": 1731472, \"loss\": 0.05365054316818714}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:05:59\", \"epochs_done\": 213, \"batches_seen\": 27300, \"train_examples_seen\": 1737828, \"loss\": 0.054217301160097126}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:00\", \"epochs_done\": 214, \"batches_seen\": 27400, \"train_examples_seen\": 1744184, \"loss\": 0.0540021326765418}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:01\", \"epochs_done\": 214, \"batches_seen\": 27500, \"train_examples_seen\": 1750584, \"loss\": 0.05379853159189224}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:50.283 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.6934\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.6911, \"roc_auc\": 0.9467}, \"time_spent\": \"0:06:02\", \"epochs_done\": 215, \"batches_seen\": 27520, \"train_examples_seen\": 1751820, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:03\", \"epochs_done\": 215, \"batches_seen\": 27600, \"train_examples_seen\": 1756940, \"loss\": 0.054332220926880836}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:05\", \"epochs_done\": 216, \"batches_seen\": 27700, \"train_examples_seen\": 1763296, \"loss\": 0.05378472100943327}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:06\", \"epochs_done\": 217, \"batches_seen\": 27800, \"train_examples_seen\": 1769652, \"loss\": 0.05444237634539604}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:07\", \"epochs_done\": 217, \"batches_seen\": 27900, \"train_examples_seen\": 1776052, \"loss\": 0.05406495194882154}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:08\", \"epochs_done\": 218, \"batches_seen\": 28000, \"train_examples_seen\": 1782408, \"loss\": 0.05366335242986679}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:09\", \"epochs_done\": 219, \"batches_seen\": 28100, \"train_examples_seen\": 1788764, \"loss\": 0.05465568505227566}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:16:58.314 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7053\n",
"2020-08-24 01:16:58.315 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:16:58.316 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7053, \"roc_auc\": 0.9483}, \"time_spent\": \"0:06:10\", \"epochs_done\": 220, \"batches_seen\": 28160, \"train_examples_seen\": 1792560, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:11\", \"epochs_done\": 220, \"batches_seen\": 28200, \"train_examples_seen\": 1795120, \"loss\": 0.05409917909651995}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:12\", \"epochs_done\": 221, \"batches_seen\": 28300, \"train_examples_seen\": 1801476, \"loss\": 0.053252529129385946}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:13\", \"epochs_done\": 221, \"batches_seen\": 28400, \"train_examples_seen\": 1807876, \"loss\": 0.053565810844302175}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:14\", \"epochs_done\": 222, \"batches_seen\": 28500, \"train_examples_seen\": 1814232, \"loss\": 0.05487801820039749}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:16\", \"epochs_done\": 223, \"batches_seen\": 28600, \"train_examples_seen\": 1820588, \"loss\": 0.05324256125837565}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:17\", \"epochs_done\": 224, \"batches_seen\": 28700, \"train_examples_seen\": 1826944, \"loss\": 0.05425327554345131}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:18\", \"epochs_done\": 224, \"batches_seen\": 28800, \"train_examples_seen\": 1833300, \"loss\": 0.05386303126811981}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:06.470 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7053\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.68, \"roc_auc\": 0.9488}, \"time_spent\": \"0:06:19\", \"epochs_done\": 225, \"batches_seen\": 28800, \"train_examples_seen\": 1833300, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:20\", \"epochs_done\": 225, \"batches_seen\": 28900, \"train_examples_seen\": 1839700, \"loss\": 0.053888755403459075}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:21\", \"epochs_done\": 226, \"batches_seen\": 29000, \"train_examples_seen\": 1846056, \"loss\": 0.053098760768771174}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:22\", \"epochs_done\": 227, \"batches_seen\": 29100, \"train_examples_seen\": 1852412, \"loss\": 0.05352992214262486}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:23\", \"epochs_done\": 228, \"batches_seen\": 29200, \"train_examples_seen\": 1858768, \"loss\": 0.05256114173680544}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:24\", \"epochs_done\": 228, \"batches_seen\": 29300, \"train_examples_seen\": 1865168, \"loss\": 0.05315491542220116}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:26\", \"epochs_done\": 229, \"batches_seen\": 29400, \"train_examples_seen\": 1871524, \"loss\": 0.05304866135120392}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:14.551 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7116\n",
"2020-08-24 01:17:14.553 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:17:14.556 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7116, \"roc_auc\": 0.9429}, \"time_spent\": \"0:06:27\", \"epochs_done\": 230, \"batches_seen\": 29440, \"train_examples_seen\": 1874040, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:27\", \"epochs_done\": 230, \"batches_seen\": 29500, \"train_examples_seen\": 1877880, \"loss\": 0.05321628995239735}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:29\", \"epochs_done\": 231, \"batches_seen\": 29600, \"train_examples_seen\": 1884236, \"loss\": 0.053719923198223114}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:30\", \"epochs_done\": 232, \"batches_seen\": 29700, \"train_examples_seen\": 1890592, \"loss\": 0.052996299490332605}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:31\", \"epochs_done\": 232, \"batches_seen\": 29800, \"train_examples_seen\": 1896992, \"loss\": 0.05323375895619392}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:32\", \"epochs_done\": 233, \"batches_seen\": 29900, \"train_examples_seen\": 1903348, \"loss\": 0.05287536058574915}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:33\", \"epochs_done\": 234, \"batches_seen\": 30000, \"train_examples_seen\": 1909704, \"loss\": 0.05303043615072966}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:22.705 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7116\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7012, \"roc_auc\": 0.9428}, \"time_spent\": \"0:06:35\", \"epochs_done\": 235, \"batches_seen\": 30080, \"train_examples_seen\": 1914780, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:35\", \"epochs_done\": 235, \"batches_seen\": 30100, \"train_examples_seen\": 1916060, \"loss\": 0.052922139130532744}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:36\", \"epochs_done\": 235, \"batches_seen\": 30200, \"train_examples_seen\": 1922460, \"loss\": 0.052376110665500165}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6875, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:37\", \"epochs_done\": 236, \"batches_seen\": 30300, \"train_examples_seen\": 1928816, \"loss\": 0.05264295894652605}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:39\", \"epochs_done\": 237, \"batches_seen\": 30400, \"train_examples_seen\": 1935172, \"loss\": 0.054487734586000446}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:40\", \"epochs_done\": 238, \"batches_seen\": 30500, \"train_examples_seen\": 1941528, \"loss\": 0.052360786087810994}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:41\", \"epochs_done\": 239, \"batches_seen\": 30600, \"train_examples_seen\": 1947884, \"loss\": 0.05175456944853067}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:42\", \"epochs_done\": 239, \"batches_seen\": 30700, \"train_examples_seen\": 1954284, \"loss\": 0.05171070877462625}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:30.806 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7116\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7104, \"roc_auc\": 0.9448}, \"time_spent\": \"0:06:43\", \"epochs_done\": 240, \"batches_seen\": 30720, \"train_examples_seen\": 1955520, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:44\", \"epochs_done\": 240, \"batches_seen\": 30800, \"train_examples_seen\": 1960640, \"loss\": 0.052260428108274935}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:45\", \"epochs_done\": 241, \"batches_seen\": 30900, \"train_examples_seen\": 1966996, \"loss\": 0.0528621768578887}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:46\", \"epochs_done\": 242, \"batches_seen\": 31000, \"train_examples_seen\": 1973352, \"loss\": 0.0514835798740387}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:47\", \"epochs_done\": 242, \"batches_seen\": 31100, \"train_examples_seen\": 1979752, \"loss\": 0.05286644145846367}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:49\", \"epochs_done\": 243, \"batches_seen\": 31200, \"train_examples_seen\": 1986108, \"loss\": 0.05257672604173422}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:50\", \"epochs_done\": 244, \"batches_seen\": 31300, \"train_examples_seen\": 1992464, \"loss\": 0.05195835389196873}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:38.988 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7382\n",
"2020-08-24 01:17:38.988 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:17:38.989 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7382, \"roc_auc\": 0.9449}, \"time_spent\": \"0:06:51\", \"epochs_done\": 245, \"batches_seen\": 31360, \"train_examples_seen\": 1996260, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:52\", \"epochs_done\": 245, \"batches_seen\": 31400, \"train_examples_seen\": 1998820, \"loss\": 0.05224948897957802}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:53\", \"epochs_done\": 246, \"batches_seen\": 31500, \"train_examples_seen\": 2005176, \"loss\": 0.051951516903936866}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:54\", \"epochs_done\": 246, \"batches_seen\": 31600, \"train_examples_seen\": 2011576, \"loss\": 0.053250179924070835}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:55\", \"epochs_done\": 247, \"batches_seen\": 31700, \"train_examples_seen\": 2017932, \"loss\": 0.051797833070158956}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:56\", \"epochs_done\": 248, \"batches_seen\": 31800, \"train_examples_seen\": 2024288, \"loss\": 0.05224585719406605}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:57\", \"epochs_done\": 249, \"batches_seen\": 31900, \"train_examples_seen\": 2030644, \"loss\": 0.05221662234514952}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7031, \"roc_auc\": 0.0}, \"time_spent\": \"0:06:59\", \"epochs_done\": 249, \"batches_seen\": 32000, \"train_examples_seen\": 2037000, \"loss\": 0.05171452037990093}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:47.150 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7382\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.947}, \"time_spent\": \"0:06:59\", \"epochs_done\": 250, \"batches_seen\": 32000, \"train_examples_seen\": 2037000, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:00\", \"epochs_done\": 250, \"batches_seen\": 32100, \"train_examples_seen\": 2043400, \"loss\": 0.05107255008071661}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:02\", \"epochs_done\": 251, \"batches_seen\": 32200, \"train_examples_seen\": 2049756, \"loss\": 0.0531261283159256}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9375, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:03\", \"epochs_done\": 252, \"batches_seen\": 32300, \"train_examples_seen\": 2056112, \"loss\": 0.05167767804116011}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:04\", \"epochs_done\": 253, \"batches_seen\": 32400, \"train_examples_seen\": 2062468, \"loss\": 0.05172851983457804}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:05\", \"epochs_done\": 253, \"batches_seen\": 32500, \"train_examples_seen\": 2068868, \"loss\": 0.050964304842054844}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:06\", \"epochs_done\": 254, \"batches_seen\": 32600, \"train_examples_seen\": 2075224, \"loss\": 0.05171708308160305}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:17:55.190 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7461\n",
"2020-08-24 01:17:55.191 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:17:55.196 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7461, \"roc_auc\": 0.947}, \"time_spent\": \"0:07:07\", \"epochs_done\": 255, \"batches_seen\": 32640, \"train_examples_seen\": 2077740, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:08\", \"epochs_done\": 255, \"batches_seen\": 32700, \"train_examples_seen\": 2081580, \"loss\": 0.051207487918436524}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7188, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:09\", \"epochs_done\": 256, \"batches_seen\": 32800, \"train_examples_seen\": 2087936, \"loss\": 0.05138891287147999}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:10\", \"epochs_done\": 257, \"batches_seen\": 32900, \"train_examples_seen\": 2094292, \"loss\": 0.051568353921175}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:12\", \"epochs_done\": 257, \"batches_seen\": 33000, \"train_examples_seen\": 2100692, \"loss\": 0.05157723594456911}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:13\", \"epochs_done\": 258, \"batches_seen\": 33100, \"train_examples_seen\": 2107048, \"loss\": 0.0506487637758255}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:14\", \"epochs_done\": 259, \"batches_seen\": 33200, \"train_examples_seen\": 2113404, \"loss\": 0.05131337508559227}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:03.294 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7461\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7417, \"roc_auc\": 0.9429}, \"time_spent\": \"0:07:15\", \"epochs_done\": 260, \"batches_seen\": 33280, \"train_examples_seen\": 2118480, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:16\", \"epochs_done\": 260, \"batches_seen\": 33300, \"train_examples_seen\": 2119760, \"loss\": 0.05217563312500715}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.6719, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:17\", \"epochs_done\": 260, \"batches_seen\": 33400, \"train_examples_seen\": 2126160, \"loss\": 0.05048427708446979}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:18\", \"epochs_done\": 261, \"batches_seen\": 33500, \"train_examples_seen\": 2132516, \"loss\": 0.05123890362679959}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:19\", \"epochs_done\": 262, \"batches_seen\": 33600, \"train_examples_seen\": 2138872, \"loss\": 0.05116777893155813}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:20\", \"epochs_done\": 263, \"batches_seen\": 33700, \"train_examples_seen\": 2145228, \"loss\": 0.051035093180835246}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:21\", \"epochs_done\": 264, \"batches_seen\": 33800, \"train_examples_seen\": 2151584, \"loss\": 0.05010158613324165}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:23\", \"epochs_done\": 264, \"batches_seen\": 33900, \"train_examples_seen\": 2157984, \"loss\": 0.05031798969954252}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:11.418 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7611\n",
"2020-08-24 01:18:11.419 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:18:11.420 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7611, \"roc_auc\": 0.9454}, \"time_spent\": \"0:07:24\", \"epochs_done\": 265, \"batches_seen\": 33920, \"train_examples_seen\": 2159220, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:25\", \"epochs_done\": 265, \"batches_seen\": 34000, \"train_examples_seen\": 2164340, \"loss\": 0.051324490793049335}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:26\", \"epochs_done\": 266, \"batches_seen\": 34100, \"train_examples_seen\": 2170696, \"loss\": 0.0503596169129014}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:27\", \"epochs_done\": 267, \"batches_seen\": 34200, \"train_examples_seen\": 2177052, \"loss\": 0.05031822204589844}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:28\", \"epochs_done\": 267, \"batches_seen\": 34300, \"train_examples_seen\": 2183452, \"loss\": 0.05045803681015968}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:29\", \"epochs_done\": 268, \"batches_seen\": 34400, \"train_examples_seen\": 2189808, \"loss\": 0.05170884713530541}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9375, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:30\", \"epochs_done\": 269, \"batches_seen\": 34500, \"train_examples_seen\": 2196164, \"loss\": 0.05030163533985615}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:19.645 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7611\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7576, \"roc_auc\": 0.949}, \"time_spent\": \"0:07:32\", \"epochs_done\": 270, \"batches_seen\": 34560, \"train_examples_seen\": 2199960, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:32\", \"epochs_done\": 270, \"batches_seen\": 34600, \"train_examples_seen\": 2202520, \"loss\": 0.05025132074952125}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:33\", \"epochs_done\": 271, \"batches_seen\": 34700, \"train_examples_seen\": 2208876, \"loss\": 0.05030454255640507}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:35\", \"epochs_done\": 271, \"batches_seen\": 34800, \"train_examples_seen\": 2215276, \"loss\": 0.05057236611843109}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:36\", \"epochs_done\": 272, \"batches_seen\": 34900, \"train_examples_seen\": 2221632, \"loss\": 0.0506666773930192}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:37\", \"epochs_done\": 273, \"batches_seen\": 35000, \"train_examples_seen\": 2227988, \"loss\": 0.04927533980458975}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:38\", \"epochs_done\": 274, \"batches_seen\": 35100, \"train_examples_seen\": 2234344, \"loss\": 0.051392757445573804}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:39\", \"epochs_done\": 274, \"batches_seen\": 35200, \"train_examples_seen\": 2240700, \"loss\": 0.048592340908944605}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:27.714 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7611\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7565, \"roc_auc\": 0.9483}, \"time_spent\": \"0:07:40\", \"epochs_done\": 275, \"batches_seen\": 35200, \"train_examples_seen\": 2240700, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:41\", \"epochs_done\": 275, \"batches_seen\": 35300, \"train_examples_seen\": 2247100, \"loss\": 0.05019298397004604}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:42\", \"epochs_done\": 276, \"batches_seen\": 35400, \"train_examples_seen\": 2253456, \"loss\": 0.05093179579824209}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:43\", \"epochs_done\": 277, \"batches_seen\": 35500, \"train_examples_seen\": 2259812, \"loss\": 0.05039779402315617}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:44\", \"epochs_done\": 278, \"batches_seen\": 35600, \"train_examples_seen\": 2266168, \"loss\": 0.0496203101426363}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:46\", \"epochs_done\": 278, \"batches_seen\": 35700, \"train_examples_seen\": 2272568, \"loss\": 0.04958374749869108}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7344, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:47\", \"epochs_done\": 279, \"batches_seen\": 35800, \"train_examples_seen\": 2278924, \"loss\": 0.04958415608853102}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:35.728 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7643\n",
"2020-08-24 01:18:35.729 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:18:35.731 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7643, \"roc_auc\": 0.9456}, \"time_spent\": \"0:07:48\", \"epochs_done\": 280, \"batches_seen\": 35840, \"train_examples_seen\": 2281440, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:49\", \"epochs_done\": 280, \"batches_seen\": 35900, \"train_examples_seen\": 2285280, \"loss\": 0.05036803670227528}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:50\", \"epochs_done\": 281, \"batches_seen\": 36000, \"train_examples_seen\": 2291636, \"loss\": 0.050721339397132394}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:51\", \"epochs_done\": 282, \"batches_seen\": 36100, \"train_examples_seen\": 2297992, \"loss\": 0.049718761406838896}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:52\", \"epochs_done\": 282, \"batches_seen\": 36200, \"train_examples_seen\": 2304392, \"loss\": 0.05095903381705284}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:53\", \"epochs_done\": 283, \"batches_seen\": 36300, \"train_examples_seen\": 2310748, \"loss\": 0.049103902615606786}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:54\", \"epochs_done\": 284, \"batches_seen\": 36400, \"train_examples_seen\": 2317104, \"loss\": 0.049757624343037606}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:43.898 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7643\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7629, \"roc_auc\": 0.9479}, \"time_spent\": \"0:07:56\", \"epochs_done\": 285, \"batches_seen\": 36480, \"train_examples_seen\": 2322180, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:56\", \"epochs_done\": 285, \"batches_seen\": 36500, \"train_examples_seen\": 2323460, \"loss\": 0.04937913581728935}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:57\", \"epochs_done\": 285, \"batches_seen\": 36600, \"train_examples_seen\": 2329860, \"loss\": 0.0497130224108696}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:07:59\", \"epochs_done\": 286, \"batches_seen\": 36700, \"train_examples_seen\": 2336216, \"loss\": 0.049935828037559984}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:00\", \"epochs_done\": 287, \"batches_seen\": 36800, \"train_examples_seen\": 2342572, \"loss\": 0.04886768370866776}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.75, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:01\", \"epochs_done\": 288, \"batches_seen\": 36900, \"train_examples_seen\": 2348928, \"loss\": 0.049990855753421784}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:02\", \"epochs_done\": 289, \"batches_seen\": 37000, \"train_examples_seen\": 2355284, \"loss\": 0.04888678699731827}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:03\", \"epochs_done\": 289, \"batches_seen\": 37100, \"train_examples_seen\": 2361684, \"loss\": 0.049254671707749366}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:18:51.991 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7643\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7565, \"roc_auc\": 0.9473}, \"time_spent\": \"0:08:04\", \"epochs_done\": 290, \"batches_seen\": 37120, \"train_examples_seen\": 2362920, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:05\", \"epochs_done\": 290, \"batches_seen\": 37200, \"train_examples_seen\": 2368040, \"loss\": 0.049716182984411716}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:06\", \"epochs_done\": 291, \"batches_seen\": 37300, \"train_examples_seen\": 2374396, \"loss\": 0.048682797364890576}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:07\", \"epochs_done\": 292, \"batches_seen\": 37400, \"train_examples_seen\": 2380752, \"loss\": 0.04944357346743345}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:09\", \"epochs_done\": 292, \"batches_seen\": 37500, \"train_examples_seen\": 2387152, \"loss\": 0.049412897974252704}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:10\", \"epochs_done\": 293, \"batches_seen\": 37600, \"train_examples_seen\": 2393508, \"loss\": 0.048864731565117836}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:11\", \"epochs_done\": 294, \"batches_seen\": 37700, \"train_examples_seen\": 2399864, \"loss\": 0.05016533561050892}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:00.71 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7661\n",
"2020-08-24 01:19:00.71 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:19:00.74 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7661, \"roc_auc\": 0.946}, \"time_spent\": \"0:08:12\", \"epochs_done\": 295, \"batches_seen\": 37760, \"train_examples_seen\": 2403660, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9219, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:13\", \"epochs_done\": 295, \"batches_seen\": 37800, \"train_examples_seen\": 2406220, \"loss\": 0.04965259078890085}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:14\", \"epochs_done\": 296, \"batches_seen\": 37900, \"train_examples_seen\": 2412576, \"loss\": 0.04878055535256862}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:15\", \"epochs_done\": 296, \"batches_seen\": 38000, \"train_examples_seen\": 2418976, \"loss\": 0.04846054408699274}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:16\", \"epochs_done\": 297, \"batches_seen\": 38100, \"train_examples_seen\": 2425332, \"loss\": 0.04954187236726284}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:17\", \"epochs_done\": 298, \"batches_seen\": 38200, \"train_examples_seen\": 2431688, \"loss\": 0.04898384150117636}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:18\", \"epochs_done\": 299, \"batches_seen\": 38300, \"train_examples_seen\": 2438044, \"loss\": 0.04804561175405979}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:20\", \"epochs_done\": 299, \"batches_seen\": 38400, \"train_examples_seen\": 2444400, \"loss\": 0.04921348724514246}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:08.193 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7735\n",
"2020-08-24 01:19:08.193 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:19:08.194 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7735, \"roc_auc\": 0.9453}, \"time_spent\": \"0:08:20\", \"epochs_done\": 300, \"batches_seen\": 38400, \"train_examples_seen\": 2444400, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:22\", \"epochs_done\": 300, \"batches_seen\": 38500, \"train_examples_seen\": 2450800, \"loss\": 0.048483316265046594}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:23\", \"epochs_done\": 301, \"batches_seen\": 38600, \"train_examples_seen\": 2457156, \"loss\": 0.04887199394404888}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7656, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:24\", \"epochs_done\": 302, \"batches_seen\": 38700, \"train_examples_seen\": 2463512, \"loss\": 0.048090325556695464}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:25\", \"epochs_done\": 303, \"batches_seen\": 38800, \"train_examples_seen\": 2469868, \"loss\": 0.04851374380290508}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:26\", \"epochs_done\": 303, \"batches_seen\": 38900, \"train_examples_seen\": 2476268, \"loss\": 0.04834264859557152}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:27\", \"epochs_done\": 304, \"batches_seen\": 39000, \"train_examples_seen\": 2482624, \"loss\": 0.0482856073975563}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:16.325 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7735\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7728, \"roc_auc\": 0.9436}, \"time_spent\": \"0:08:28\", \"epochs_done\": 305, \"batches_seen\": 39040, \"train_examples_seen\": 2485140, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:29\", \"epochs_done\": 305, \"batches_seen\": 39100, \"train_examples_seen\": 2488980, \"loss\": 0.04883166398853064}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9219, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:30\", \"epochs_done\": 306, \"batches_seen\": 39200, \"train_examples_seen\": 2495336, \"loss\": 0.048237425796687605}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:31\", \"epochs_done\": 307, \"batches_seen\": 39300, \"train_examples_seen\": 2501692, \"loss\": 0.04811892323195934}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:33\", \"epochs_done\": 307, \"batches_seen\": 39400, \"train_examples_seen\": 2508092, \"loss\": 0.048177977055311204}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:34\", \"epochs_done\": 308, \"batches_seen\": 39500, \"train_examples_seen\": 2514448, \"loss\": 0.04867682591080666}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:35\", \"epochs_done\": 309, \"batches_seen\": 39600, \"train_examples_seen\": 2520804, \"loss\": 0.04833348836749792}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:24.381 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 207: Improved best sets_accuracy of 0.7813\n",
"2020-08-24 01:19:24.382 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 209: Saving model\n",
"2020-08-24 01:19:24.383 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 346: [saving model to /root/.deeppavlov/models/classifiers/intents_dstc2_v10/model_opt.json]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7813, \"roc_auc\": 0.9502}, \"time_spent\": \"0:08:37\", \"epochs_done\": 310, \"batches_seen\": 39680, \"train_examples_seen\": 2525880, \"impatience\": 0, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:37\", \"epochs_done\": 310, \"batches_seen\": 39700, \"train_examples_seen\": 2527160, \"loss\": 0.04794008865952492}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:38\", \"epochs_done\": 310, \"batches_seen\": 39800, \"train_examples_seen\": 2533560, \"loss\": 0.04835477419197559}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:39\", \"epochs_done\": 311, \"batches_seen\": 39900, \"train_examples_seen\": 2539916, \"loss\": 0.04852351114153862}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:40\", \"epochs_done\": 312, \"batches_seen\": 40000, \"train_examples_seen\": 2546272, \"loss\": 0.04693978484719992}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9375, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:41\", \"epochs_done\": 313, \"batches_seen\": 40100, \"train_examples_seen\": 2552628, \"loss\": 0.0479451860114932}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:43\", \"epochs_done\": 314, \"batches_seen\": 40200, \"train_examples_seen\": 2558984, \"loss\": 0.048769728280603886}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:44\", \"epochs_done\": 314, \"batches_seen\": 40300, \"train_examples_seen\": 2565384, \"loss\": 0.04823755450546741}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:32.467 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7813\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7762, \"roc_auc\": 0.9495}, \"time_spent\": \"0:08:45\", \"epochs_done\": 315, \"batches_seen\": 40320, \"train_examples_seen\": 2566620, \"impatience\": 1, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:46\", \"epochs_done\": 315, \"batches_seen\": 40400, \"train_examples_seen\": 2571740, \"loss\": 0.04731459580361843}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:47\", \"epochs_done\": 316, \"batches_seen\": 40500, \"train_examples_seen\": 2578096, \"loss\": 0.047941643297672275}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:48\", \"epochs_done\": 317, \"batches_seen\": 40600, \"train_examples_seen\": 2584452, \"loss\": 0.04804954320192337}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:49\", \"epochs_done\": 317, \"batches_seen\": 40700, \"train_examples_seen\": 2590852, \"loss\": 0.047543941624462606}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:50\", \"epochs_done\": 318, \"batches_seen\": 40800, \"train_examples_seen\": 2597208, \"loss\": 0.047564432546496394}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:51\", \"epochs_done\": 319, \"batches_seen\": 40900, \"train_examples_seen\": 2603564, \"loss\": 0.049497693143785}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:40.643 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7813\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7723, \"roc_auc\": 0.9459}, \"time_spent\": \"0:08:53\", \"epochs_done\": 320, \"batches_seen\": 40960, \"train_examples_seen\": 2607360, \"impatience\": 2, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:53\", \"epochs_done\": 320, \"batches_seen\": 41000, \"train_examples_seen\": 2609920, \"loss\": 0.0468096555583179}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9219, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:54\", \"epochs_done\": 321, \"batches_seen\": 41100, \"train_examples_seen\": 2616276, \"loss\": 0.04732314016669989}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:56\", \"epochs_done\": 321, \"batches_seen\": 41200, \"train_examples_seen\": 2622676, \"loss\": 0.04757677219808101}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:57\", \"epochs_done\": 322, \"batches_seen\": 41300, \"train_examples_seen\": 2629032, \"loss\": 0.04661393191665411}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:58\", \"epochs_done\": 323, \"batches_seen\": 41400, \"train_examples_seen\": 2635388, \"loss\": 0.04778644923120737}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:08:59\", \"epochs_done\": 324, \"batches_seen\": 41500, \"train_examples_seen\": 2641744, \"loss\": 0.046841290667653086}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:00\", \"epochs_done\": 324, \"batches_seen\": 41600, \"train_examples_seen\": 2648100, \"loss\": 0.04746122676879168}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:48.959 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7813\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7684, \"roc_auc\": 0.9463}, \"time_spent\": \"0:09:01\", \"epochs_done\": 325, \"batches_seen\": 41600, \"train_examples_seen\": 2648100, \"impatience\": 3, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:02\", \"epochs_done\": 325, \"batches_seen\": 41700, \"train_examples_seen\": 2654500, \"loss\": 0.046944671645760536}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:03\", \"epochs_done\": 326, \"batches_seen\": 41800, \"train_examples_seen\": 2660856, \"loss\": 0.0475559002161026}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8906, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:05\", \"epochs_done\": 327, \"batches_seen\": 41900, \"train_examples_seen\": 2667212, \"loss\": 0.04786529827862978}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8438, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:06\", \"epochs_done\": 328, \"batches_seen\": 42000, \"train_examples_seen\": 2673568, \"loss\": 0.04664505604654551}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.875, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:07\", \"epochs_done\": 328, \"batches_seen\": 42100, \"train_examples_seen\": 2679968, \"loss\": 0.04732142943888903}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7812, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:08\", \"epochs_done\": 329, \"batches_seen\": 42200, \"train_examples_seen\": 2686324, \"loss\": 0.04629994297400117}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:19:57.77 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7813\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7725, \"roc_auc\": 0.9458}, \"time_spent\": \"0:09:09\", \"epochs_done\": 330, \"batches_seen\": 42240, \"train_examples_seen\": 2688840, \"impatience\": 4, \"patience_limit\": 5}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.7969, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:10\", \"epochs_done\": 330, \"batches_seen\": 42300, \"train_examples_seen\": 2692680, \"loss\": 0.04758493658155203}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.9062, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:11\", \"epochs_done\": 331, \"batches_seen\": 42400, \"train_examples_seen\": 2699036, \"loss\": 0.047077034674584864}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:12\", \"epochs_done\": 332, \"batches_seen\": 42500, \"train_examples_seen\": 2705392, \"loss\": 0.04776582509279251}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8594, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:13\", \"epochs_done\": 332, \"batches_seen\": 42600, \"train_examples_seen\": 2711792, \"loss\": 0.04692811980843544}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8281, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:15\", \"epochs_done\": 333, \"batches_seen\": 42700, \"train_examples_seen\": 2718148, \"loss\": 0.046551134027540686}}\n",
"{\"train\": {\"eval_examples_count\": 64, \"metrics\": {\"sets_accuracy\": 0.8125, \"roc_auc\": 0.0}, \"time_spent\": \"0:09:16\", \"epochs_done\": 334, \"batches_seen\": 42800, \"train_examples_seen\": 2724504, \"loss\": 0.04685700871050358}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:20:05.154 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 212: Did not improve on the sets_accuracy of 0.7813\n",
"2020-08-24 01:20:05.157 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 329: Ran out of patience\n",
"2020-08-24 01:20:05.185 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/.deeppavlov/models/classifiers/intents_dstc2_v10/classes.dict]\n",
"2020-08-24 01:20:05.187 INFO in 'deeppavlov.models.embedders.fasttext_embedder'['fasttext_embedder'] at line 53: [loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/dstc2_fastText_model.bin`]\n",
"\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7772, \"roc_auc\": 0.9441}, \"time_spent\": \"0:09:17\", \"epochs_done\": 335, \"batches_seen\": 42880, \"train_examples_seen\": 2729580, \"impatience\": 5, \"patience_limit\": 5}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:20:05.735 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 245: [initializing `KerasClassificationModel` from saved]\n",
"2020-08-24 01:20:06.114 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 255: [loading weights from model.h5]\n",
"2020-08-24 01:20:06.362 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 129: Model was successfully initialized!\n",
"Model summary:\n",
"Model: \"model\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_1 (InputLayer) [(None, None, 100)] 0 \n",
"__________________________________________________________________________________________________\n",
"conv1d (Conv1D) (None, None, 512) 154112 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_1 (Conv1D) (None, None, 512) 256512 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_2 (Conv1D) (None, None, 512) 358912 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization (BatchNorma (None, None, 512) 2048 conv1d[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_1 (BatchNor (None, None, 512) 2048 conv1d_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_2 (BatchNor (None, None, 512) 2048 conv1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation (Activation) (None, None, 512) 0 batch_normalization[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_1 (Activation) (None, None, 512) 0 batch_normalization_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_2 (Activation) (None, None, 512) 0 batch_normalization_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d (GlobalMax (None, 512) 0 activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_1 (GlobalM (None, 512) 0 activation_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_2 (GlobalM (None, 512) 0 activation_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"concatenate (Concatenate) (None, 1536) 0 global_max_pooling1d[0][0] \n",
" global_max_pooling1d_1[0][0] \n",
" global_max_pooling1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout (Dropout) (None, 1536) 0 concatenate[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense (Dense) (None, 100) 153700 dropout[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_3 (BatchNor (None, 100) 400 dense[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_3 (Activation) (None, 100) 0 batch_normalization_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout_1 (Dropout) (None, 100) 0 activation_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_1 (Dense) (None, 28) 2828 dropout_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_4 (BatchNor (None, 28) 112 dense_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_4 (Activation) (None, 28) 0 batch_normalization_4[0][0] \n",
"==================================================================================================\n",
"Total params: 932,720\n",
"Trainable params: 929,392\n",
"Non-trainable params: 3,328\n",
"__________________________________________________________________________________________________\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"train\": {\"eval_examples_count\": 8148, \"metrics\": {\"sets_accuracy\": 0.8447, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:02\"}}\n",
"{\"valid\": {\"eval_examples_count\": 5656, \"metrics\": {\"sets_accuracy\": 0.7813, \"roc_auc\": 0.9502}, \"time_spent\": \"0:00:01\"}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"2020-08-24 01:20:09.263 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/.deeppavlov/models/classifiers/intents_dstc2_v10/classes.dict]\n",
"2020-08-24 01:20:09.264 INFO in 'deeppavlov.models.embedders.fasttext_embedder'['fasttext_embedder'] at line 53: [loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/dstc2_fastText_model.bin`]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{\"test\": {\"eval_examples_count\": 5769, \"metrics\": {\"sets_accuracy\": 0.7875, \"roc_auc\": 0.0}, \"time_spent\": \"0:00:01\"}}\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"\n",
"2020-08-24 01:20:09.704 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 245: [initializing `KerasClassificationModel` from saved]\n",
"2020-08-24 01:20:10.74 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 255: [loading weights from model.h5]\n",
"2020-08-24 01:20:10.344 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 129: Model was successfully initialized!\n",
"Model summary:\n",
"Model: \"model\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_1 (InputLayer) [(None, None, 100)] 0 \n",
"__________________________________________________________________________________________________\n",
"conv1d (Conv1D) (None, None, 512) 154112 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_1 (Conv1D) (None, None, 512) 256512 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_2 (Conv1D) (None, None, 512) 358912 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization (BatchNorma (None, None, 512) 2048 conv1d[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_1 (BatchNor (None, None, 512) 2048 conv1d_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_2 (BatchNor (None, None, 512) 2048 conv1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation (Activation) (None, None, 512) 0 batch_normalization[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_1 (Activation) (None, None, 512) 0 batch_normalization_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_2 (Activation) (None, None, 512) 0 batch_normalization_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d (GlobalMax (None, 512) 0 activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_1 (GlobalM (None, 512) 0 activation_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_2 (GlobalM (None, 512) 0 activation_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"concatenate (Concatenate) (None, 1536) 0 global_max_pooling1d[0][0] \n",
" global_max_pooling1d_1[0][0] \n",
" global_max_pooling1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout (Dropout) (None, 1536) 0 concatenate[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense (Dense) (None, 100) 153700 dropout[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_3 (BatchNor (None, 100) 400 dense[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_3 (Activation) (None, 100) 0 batch_normalization_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout_1 (Dropout) (None, 100) 0 activation_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_1 (Dense) (None, 28) 2828 dropout_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_4 (BatchNor (None, 28) 112 dense_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_4 (Activation) (None, 28) 0 batch_normalization_4[0][0] \n",
"==================================================================================================\n",
"Total params: 932,720\n",
"Trainable params: 929,392\n",
"Non-trainable params: 3,328\n",
"__________________________________________________________________________________________________\n"
],
"name": "stderr"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"Chainer['classes_vocab': <deeppavlov.core.data.simple_vocab.SimpleVocabulary at 0x7f05884ec518>,\n",
" 'my_tokenizer': <deeppavlov.models.tokenizers.nltk_tokenizer.NLTKTokenizer at 0x7f0530b5c128>,\n",
" 'my_embedder': <deeppavlov.models.embedders.fasttext_embedder.FasttextEmbedder at 0x7f059c417f28>,\n",
" 'my_one_hotter': <deeppavlov.models.preprocessors.one_hotter.OneHotter at 0x7f059ad728d0>,\n",
" <deeppavlov.models.classifiers.keras_classification_model.KerasClassificationModel at 0x7f059c42a978>,\n",
" <deeppavlov.models.classifiers.proba2labels.Proba2Labels at 0x7f0518330208>,\n",
" 'classes_vocab': <deeppavlov.core.data.simple_vocab.SimpleVocabulary at 0x7f05884ec518>,\n",
" 'my_one_hotter': <deeppavlov.models.preprocessors.one_hotter.OneHotter at 0x7f059ad728d0>]"
]
},
"metadata": {
"tags": []
},
"execution_count": 9
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y2wT2lq2BA5U",
"colab_type": "text"
},
"source": [
"### inspect the trained model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "q5NwMO6F_U2a",
"colab_type": "text"
},
"source": [
"Let's load the trained model and check whether it works OK."
]
},
{
"cell_type": "code",
"metadata": {
"id": "CWGf-FmJxG3q",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "cfbe5564-e893-4322-c222-64a6638439bb"
},
"source": [
"intents_classifier = build_model(intents_clf_config_di)"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"text": [
"2020-08-24 01:20:10.360 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/.deeppavlov/models/classifiers/intents_dstc2_v10/classes.dict]\n",
"2020-08-24 01:20:10.368 INFO in 'deeppavlov.models.embedders.fasttext_embedder'['fasttext_embedder'] at line 53: [loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/dstc2_fastText_model.bin`]\n",
"\n",
"2020-08-24 01:20:11.106 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 245: [initializing `KerasClassificationModel` from saved]\n",
"2020-08-24 01:20:11.473 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 255: [loading weights from model.h5]\n",
"2020-08-24 01:20:11.727 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 129: Model was successfully initialized!\n",
"Model summary:\n",
"Model: \"model\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_1 (InputLayer) [(None, None, 100)] 0 \n",
"__________________________________________________________________________________________________\n",
"conv1d (Conv1D) (None, None, 512) 154112 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_1 (Conv1D) (None, None, 512) 256512 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv1d_2 (Conv1D) (None, None, 512) 358912 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization (BatchNorma (None, None, 512) 2048 conv1d[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_1 (BatchNor (None, None, 512) 2048 conv1d_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_2 (BatchNor (None, None, 512) 2048 conv1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation (Activation) (None, None, 512) 0 batch_normalization[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_1 (Activation) (None, None, 512) 0 batch_normalization_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_2 (Activation) (None, None, 512) 0 batch_normalization_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d (GlobalMax (None, 512) 0 activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_1 (GlobalM (None, 512) 0 activation_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling1d_2 (GlobalM (None, 512) 0 activation_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"concatenate (Concatenate) (None, 1536) 0 global_max_pooling1d[0][0] \n",
" global_max_pooling1d_1[0][0] \n",
" global_max_pooling1d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout (Dropout) (None, 1536) 0 concatenate[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense (Dense) (None, 100) 153700 dropout[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_3 (BatchNor (None, 100) 400 dense[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_3 (Activation) (None, 100) 0 batch_normalization_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout_1 (Dropout) (None, 100) 0 activation_3[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_1 (Dense) (None, 28) 2828 dropout_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"batch_normalization_4 (BatchNor (None, 28) 112 dense_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"activation_4 (Activation) (None, 28) 0 batch_normalization_4[0][0] \n",
"==================================================================================================\n",
"Total params: 932,720\n",
"Trainable params: 929,392\n",
"Non-trainable params: 3,328\n",
"__________________________________________________________________________________________________\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "JEULv3LD8qqh",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 136
},
"outputId": "58193ad5-28ac-4ddb-d57a-29b59952b3f2"
},
"source": [
"intents_classifier([\"ok thank you bye!\"])"
],
"execution_count": 11,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[[['bye', 'thankyou']],\n",
" array([[0.00241262, 0.03746465, 0.03026664, 0.02149013, 0.9481962 ,\n",
" 0.8559191 , 0.02318546, 0.01851559, 0.03142855, 0.01543504,\n",
" 0.01435608, 0.01208022, 0.0059303 , 0.00974587, 0.01084879,\n",
" 0.0091615 , 0.01057598, 0.00583509, 0.00482801, 0.00447744,\n",
" 0.00429937, 0.00369725, 0.00324008, 0.00322673, 0.00293323,\n",
" 0.00311375, 0.00287911, 0.00258186]], dtype=float32)]"
]
},
"metadata": {
"tags": []
},
"execution_count": 11
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uIR-8B9fBDiW",
"colab_type": "text"
},
"source": [
"### sandbox"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HaVqT0xn_Zfg",
"colab_type": "text"
},
"source": [
"Below is the dummy cell which is useful to be queued to run: it prevents colab from restart."
]
},
{
"cell_type": "code",
"metadata": {
"id": "EMGR5a6D_jet",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 340
},
"outputId": "9166fe8f-8415-4153-8a7b-1c3c4229a840"
},
"source": [
"import time\n",
"\n",
"while True:\n",
" time.sleep(2)\n",
" !echo $(date)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Mon Aug 24 01:20:14 UTC 2020\n",
"Mon Aug 24 01:20:17 UTC 2020\n",
"Mon Aug 24 01:20:20 UTC 2020\n",
"Mon Aug 24 01:20:23 UTC 2020\n",
"Mon Aug 24 01:20:26 UTC 2020\n",
"Mon Aug 24 01:20:29 UTC 2020\n",
"Mon Aug 24 01:20:32 UTC 2020\n",
"Mon Aug 24 01:20:35 UTC 2020\n",
"Mon Aug 24 01:20:38 UTC 2020\n",
"Mon Aug 24 01:20:42 UTC 2020\n",
"Mon Aug 24 01:20:45 UTC 2020\n",
"Mon Aug 24 01:20:48 UTC 2020\n",
"Mon Aug 24 01:20:51 UTC 2020\n",
"Mon Aug 24 01:20:54 UTC 2020\n",
"Mon Aug 24 01:20:57 UTC 2020\n",
"Mon Aug 24 01:21:00 UTC 2020\n",
"Mon Aug 24 01:21:03 UTC 2020\n",
"Mon Aug 24 01:21:07 UTC 2020\n",
"Mon Aug 24 01:21:10 UTC 2020\n"
],
"name": "stdout"
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment