projetTal1_classification_prenoms

01_classification_prenoms.ipynb

{
  "cells": [
    {
      "metadata": {
        "pycharm": {
          "name": "#%% md\n"
        }
      },
      "cell_type": "markdown",
      "source": "# Classification de prénoms en genre (masculin/féminin)\n\nCet exemple très simple issu du [livre](https://www.amazon.fr/Natural-Language-Processing-Python-Steven/dp/0596516495/ref=sr_1_11?__mk_fr_FR=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=nltk&qid=1616280931&sr=8-11) de Steven Bird vous donne une idée sur à quoi ressemble le boulot d'un ingénieur junior en Machine Learning.\n\nLa tâche consiste à entraîner un classifieur bayésien pour prédire le genre d'un prénom.\n"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 1. Sélection de features\n\nOn commence par prendre la dernière lettre d'un prénom comme feature et la stocker dans un dictionnaire."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2021-03-12T13:57:46.133538Z",
          "start_time": "2021-03-12T13:57:46.130908Z"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! creation de last latter comme feature\n\nimport nltk\nfrom nltk.corpus import names\nimport random\nrandom.seed(13)\ndef gender_features(word):\n     return {'last_letter': word[-1]}\n\nprint(\"La dernière lettre du pronom Shrek est\")\ngender_features('Shrek')",
      "execution_count": 3,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "La dernière lettre du prnom Shrek est\n"
        },
        {
          "data": {
            "text/plain": "{'last_letter': 'k'}"
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 2. Mise en forme du corpus\n\nLe corpus provient de `nltk`. Ici on crée une liste de tuples grâce à quelques méthodes intégrées dans `nltk.corpus`. Notez l'emploi de list comprehension ici pour rendre le code plus concis tout en gardant la lisibilité."
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2021-03-12T13:59:12.603355Z",
          "start_time": "2021-03-12T13:59:11.440907Z"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! creation de datasets\n\nlabeled_names = ([(name, 'male') for name in names.words('male.txt')] +\n [(name, 'female') for name in names.words('female.txt')])\n\nrandom.shuffle(labeled_names)\nprint(\"Un échantillon du corpus\")\nlabeled_names[:10]",
      "execution_count": 6,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "Un échantillon du corpus\n"
        },
        {
          "data": {
            "text/plain": "[('Mariam', 'female'),\n ('Marjorie', 'female'),\n ('Jasmin', 'female'),\n ('Welbie', 'male'),\n ('Modesty', 'female'),\n ('Kanya', 'female'),\n ('Michale', 'male'),\n ('Antonina', 'female'),\n ('Beulah', 'female'),\n ('Hazel', 'female')]"
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 3. Création des corpus train/test\n\nIci on applique à la fonction de la `section 1` à tous les noms du corpus. Les 500 premiers samples sont mis à l'écart pour servir de test."
    },
    {
      "metadata": {
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! creation de paire feature/label\n\nfeaturesets = [(gender_features(n), gender) for (n, gender) in labeled_names]",
      "execution_count": 10,
      "outputs": []
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! creation de train et test\n\ntrain_set, test_set = featuresets[500:], featuresets[:500]",
      "execution_count": 11,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 4. Première classification\n\nLa précision est autour de 75.2%. On liste les features les plus utiles pour étudier quels sont les problèmes potentiels. Le likely ratio `male : female` signifie la probabilité exacte qu'un prénom particulier soit masculin/féminin en fonction de sa dernière lettre (donc feature)."
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "classifier = nltk.NaiveBayesClassifier.train(train_set)\n#! classify\nclassifier.classify(gender_features('Neo'))\nclassifier.classify(gender_features('Trinity'))\nprint(nltk.classify.accuracy(classifier, test_set))\nclassifier.show_most_informative_features(5)\n#! see likely ratio",
      "execution_count": 12,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "0.752\nMost Informative Features\n             last_letter = 'k'              male : female =     45.8 : 1.0\n             last_letter = 'a'            female : male   =     33.0 : 1.0\n             last_letter = 'f'              male : female =     15.3 : 1.0\n             last_letter = 'p'              male : female =     11.2 : 1.0\n             last_letter = 'v'              male : female =     10.5 : 1.0\n"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 5. Ajout de features et problème d'overfitting\n\nSi vous ajoutez trop de features, le modèle risque d'être trop adapté à tes données et se généralise mal sur des données non vues. Cela s'appelle `overfitting` et survient souvent quand le corpus est petit, ce qui est le cas ici."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 6. Let's add features\n\nDans un premier temps, nous allons essayer d'ajouter plein de features.\n\nExaminons la fonction `gender_features2`. Les features sont :\n\n* La première et la dernière lettre\n* Un booléen indiquant si une lettre de l'ensemble a-z est présent dans le prénom\n* Un integer indiquant le nombre d'occurrences de cette lettre\n\nDonc clairement nous y mettons tout le paquet..."
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! add features\ndef gender_features2(name):\n    features = {\"first_letter\": name[0].lower(), \"last_letter\": name[-1].lower()}\n    for letter in 'abcdefghijklmnopqrstuvwxyz':\n        features[\"count({})\".format(letter)] = name.lower().count(letter)\n        features[\"has({})\".format(letter)] = (letter in name.lower())\n    return features\n\ndemo = gender_features2(\"john\")\nrandom.sample(demo.items(),5)",
      "execution_count": 13,
      "outputs": [
        {
          "data": {
            "text/plain": "[('count(i)', 0),\n ('count(h)', 1),\n ('count(x)', 0),\n ('has(a)', False),\n ('has(h)', True)]"
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 7. La précision augmente\n\nAvec ce nouveau featureset, la précision est montée de 75.2% à 77.4%."
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "featuresets = [(gender_features2(n), gender) for (n, gender) in labeled_names]\ntrain_set, test_set = featuresets[500:], featuresets[:500]\nclassifier = nltk.NaiveBayesClassifier.train(train_set)\nprint(nltk.classify.accuracy(classifier, test_set))\n\n#! 0.752 vs 0.774",
      "execution_count": 14,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "0.774\n"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 8. Feature engineering via l'analyse des erreurs\n\nIci nous expliquons le processus de feature engineering qui consiste à analyser les erreurs de la machine sur la base desquelles on filtre/supprime/créé des features.\n\nL'intelligence artificielle n'est finalement pas si artificielle, non ? (Bon j'avoue que c'est pas du deep learning, mais quand même)\n\nNotons ici la création de 'devset'. Cette répartition en 3 sets est canonique en Machine Learning. On utilise le train pour entraîner le modèle, le devset pour ajuster ce modèle. Enfin le test ne doit être utilisé que pour l'évaluation finale.\n\nLa précision initiale (avant le feature engineering) est donc 76%."
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "train_names = labeled_names[1500:]\ndevtest_names = labeled_names[500:1500]\ntest_names = labeled_names[:500]",
      "execution_count": 15,
      "outputs": []
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "#! use devtest\ntrain_set = [(gender_features(n), gender) for (n, gender) in train_names]\ndevtest_set = [(gender_features(n), gender) for (n, gender) in devtest_names]\ntest_set = [(gender_features(n), gender) for (n, gender) in test_names]\nclassifier = nltk.NaiveBayesClassifier.train(train_set)\n\nprint(nltk.classify.accuracy(classifier, devtest_set))",
      "execution_count": 16,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "0.76\n"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## 9. Premières hypothèses sur les erreurs\n\nAnalysons les erreurs affichées ci-dessous.\n\nLes pronoms terminés par `yn` tendent à être féminins, alors que ceux terminés par `n` tendent à être masculins. Du coup deux règles serait meilleures qu'une seule.\n\nCa semble être le même principe pour les pronoms terminés par `h` qui sont principalement féminins et ceux terminés par `ch` qui ont tendance à être masculins"
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "errors = []\nfor (name, tag) in devtest_names:\n     guess = classifier.classify(gender_features(name))\n     if guess != tag:\n         errors.append( (tag, guess, name) )\n\nfor (tag, guess, name) in sorted(errors):\n     print('correct={:<8} guess={:<8s} name={:<30}'.format(tag, guess, name))",
      "execution_count": 17,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "correct=female   guess=male     name=Aeriel                        \ncorrect=female   guess=male     name=Aeriell                       \ncorrect=female   guess=male     name=Allis                         \ncorrect=female   guess=male     name=Allsun                        \ncorrect=female   guess=male     name=Allyn                         \ncorrect=female   guess=male     name=Allys                         \ncorrect=female   guess=male     name=Amargo                        \ncorrect=female   guess=male     name=Amber                         \ncorrect=female   guess=male     name=Anne-Mar                      \ncorrect=female   guess=male     name=Aurel                         \ncorrect=female   guess=male     name=Avril                         \ncorrect=female   guess=male     name=Barb                          \ncorrect=female   guess=male     name=Beatriz                       \ncorrect=female   guess=male     name=Beilul                        \ncorrect=female   guess=male     name=Calypso                       \ncorrect=female   guess=male     name=Cameo                         \ncorrect=female   guess=male     name=Carlin                        \ncorrect=female   guess=male     name=Carol                         \ncorrect=female   guess=male     name=Carol-Jean                    \ncorrect=female   guess=male     name=Caron                         \ncorrect=female   guess=male     name=Caryl                         \ncorrect=female   guess=male     name=Cat                           \ncorrect=female   guess=male     name=Ceil                          \ncorrect=female   guess=male     name=Charin                        \ncorrect=female   guess=male     name=Charleen                      \ncorrect=female   guess=male     name=Charlott                      \ncorrect=female   guess=male     name=Charmian                      \ncorrect=female   guess=male     name=Charo                         \ncorrect=female   guess=male     name=Christal                      \ncorrect=female   guess=male     name=Christel                      \ncorrect=female   guess=male     name=Cleo                          \ncorrect=female   guess=male     name=Corliss                       \ncorrect=female   guess=male     name=Cris                          \ncorrect=female   guess=male     name=Cristabel                     \ncorrect=female   guess=male     name=Cybill                        \ncorrect=female   guess=male     name=Dael                          \ncorrect=female   guess=male     name=Daloris                       \ncorrect=female   guess=male     name=Darb                          \ncorrect=female   guess=male     name=Del                           \ncorrect=female   guess=male     name=Delores                       \ncorrect=female   guess=male     name=Dian                          \ncorrect=female   guess=male     name=Doloritas                     \ncorrect=female   guess=male     name=Dorcas                        \ncorrect=female   guess=male     name=Doreen                        \ncorrect=female   guess=male     name=Dorian                        \ncorrect=female   guess=male     name=Estell                        \ncorrect=female   guess=male     name=Esther                        \ncorrect=female   guess=male     name=Felicdad                      \ncorrect=female   guess=male     name=Gael                          \ncorrect=female   guess=male     name=Gilligan                      \ncorrect=female   guess=male     name=Gladys                        \ncorrect=female   guess=male     name=Glen                          \ncorrect=female   guess=male     name=Glynis                        \ncorrect=female   guess=male     name=Greer                         \ncorrect=female   guess=male     name=Grissel                       \ncorrect=female   guess=male     name=Heather                       \ncorrect=female   guess=male     name=Helen                         \ncorrect=female   guess=male     name=Hildegaard                    \ncorrect=female   guess=male     name=Ingeborg                      \ncorrect=female   guess=male     name=Iseabal                       \ncorrect=female   guess=male     name=Jaclin                        \ncorrect=female   guess=male     name=Janel                         \ncorrect=female   guess=male     name=Jen                           \ncorrect=female   guess=male     name=Jenifer                       \ncorrect=female   guess=male     name=Jo-Ann                        \ncorrect=female   guess=male     name=Jolyn                         \ncorrect=female   guess=male     name=Jolynn                        \ncorrect=female   guess=male     name=Joyan                         \ncorrect=female   guess=male     name=Kaitlynn                      \ncorrect=female   guess=male     name=Karilynn                      \ncorrect=female   guess=male     name=Kass                          \ncorrect=female   guess=male     name=Kathlin                       \ncorrect=female   guess=male     name=Kristan                       \ncorrect=female   guess=male     name=Kristen                       \ncorrect=female   guess=male     name=Lilian                        \ncorrect=female   guess=male     name=Lurleen                       \ncorrect=female   guess=male     name=Lynnett                       \ncorrect=female   guess=male     name=Madlen                        \ncorrect=female   guess=male     name=Margot                        \ncorrect=female   guess=male     name=Margret                       \ncorrect=female   guess=male     name=Mariam                        \ncorrect=female   guess=male     name=Marie-Ann                     \ncorrect=female   guess=male     name=Mariel                        \ncorrect=female   guess=male     name=Marilyn                       \ncorrect=female   guess=male     name=Maris                         \ncorrect=female   guess=male     name=Mead                          \ncorrect=female   guess=male     name=Meg                           \ncorrect=female   guess=male     name=Megen                         \ncorrect=female   guess=male     name=Meggan                        \ncorrect=female   guess=male     name=Meridel                       \ncorrect=female   guess=male     name=Mildred                       \ncorrect=female   guess=male     name=Moll                          \ncorrect=female   guess=male     name=Nell                          \ncorrect=female   guess=male     name=Noellyn                       \ncorrect=female   guess=male     name=Peg                           \ncorrect=female   guess=male     name=Persis                        \ncorrect=female   guess=male     name=Phil                          \ncorrect=female   guess=male     name=Piper                         \ncorrect=female   guess=male     name=Quinn                         \ncorrect=female   guess=male     name=Robbin                        \ncorrect=female   guess=male     name=Rosabel                       \ncorrect=female   guess=male     name=Rosaleen                      \ncorrect=female   guess=male     name=Rosalyn                       \ncorrect=female   guess=male     name=Sal                           \ncorrect=female   guess=male     name=Sara-Ann                      \ncorrect=female   guess=male     name=Shannon                       \ncorrect=female   guess=male     name=Sharyl                        \ncorrect=female   guess=male     name=Shell                         \ncorrect=female   guess=male     name=Starlin                       \ncorrect=female   guess=male     name=Theo                          \ncorrect=female   guess=male     name=Tiff                          \ncorrect=female   guess=male     name=Vivyan                        \ncorrect=female   guess=male     name=Willow                        \ncorrect=female   guess=male     name=Willyt                        \ncorrect=female   guess=male     name=Yehudit                       \ncorrect=male     guess=female   name=Abdullah                      \ncorrect=male     guess=female   name=Amory                         \ncorrect=male     guess=female   name=Angie                         \ncorrect=male     guess=female   name=Arne                          \ncorrect=male     guess=female   name=Ash                           \ncorrect=male     guess=female   name=Aube                          \ncorrect=male     guess=female   name=Aubrey                        \ncorrect=male     guess=female   name=Augie                         \ncorrect=male     guess=female   name=Baillie                       \ncorrect=male     guess=female   name=Bartholemy                    \ncorrect=male     guess=female   name=Bary                          \ncorrect=male     guess=female   name=Benjy                         \ncorrect=male     guess=female   name=Berke                         \ncorrect=male     guess=female   name=Berkley                       \ncorrect=male     guess=female   name=Boniface                      \ncorrect=male     guess=female   name=Boyce                         \ncorrect=male     guess=female   name=Bruce                         \ncorrect=male     guess=female   name=Carleigh                      \ncorrect=male     guess=female   name=Chevy                         \ncorrect=male     guess=female   name=Clancy                        \ncorrect=male     guess=female   name=Cobbie                        \ncorrect=male     guess=female   name=Cole                          \ncorrect=male     guess=female   name=Constantine                   \ncorrect=male     guess=female   name=Courtney                      \ncorrect=male     guess=female   name=Davidde                       \ncorrect=male     guess=female   name=Dudley                        \ncorrect=male     guess=female   name=Duffie                        \ncorrect=male     guess=female   name=Durante                       \ncorrect=male     guess=female   name=Eddie                         \ncorrect=male     guess=female   name=Eddy                          \ncorrect=male     guess=female   name=Erny                          \ncorrect=male     guess=female   name=Fairfax                       \ncorrect=male     guess=female   name=Felice                        \ncorrect=male     guess=female   name=Filipe                        \ncorrect=male     guess=female   name=Fonsie                        \ncorrect=male     guess=female   name=Freddie                       \ncorrect=male     guess=female   name=Gerry                         \ncorrect=male     guess=female   name=Godfrey                       \ncorrect=male     guess=female   name=Guthrie                       \ncorrect=male     guess=female   name=Guthry                        \ncorrect=male     guess=female   name=Haleigh                       \ncorrect=male     guess=female   name=Hamish                        \ncorrect=male     guess=female   name=Harvey                        \ncorrect=male     guess=female   name=Hersh                         \ncorrect=male     guess=female   name=Hodge                         \ncorrect=male     guess=female   name=Hy                            \ncorrect=male     guess=female   name=Isadore                       \ncorrect=male     guess=female   name=Jean-Pierre                   \ncorrect=male     guess=female   name=Jeffery                       \ncorrect=male     guess=female   name=Jeromy                        \ncorrect=male     guess=female   name=Jesse                         \ncorrect=male     guess=female   name=Johnny                        \ncorrect=male     guess=female   name=Josh                          \ncorrect=male     guess=female   name=Jule                          \ncorrect=male     guess=female   name=Julie                         \ncorrect=male     guess=female   name=Kelly                         \ncorrect=male     guess=female   name=Kelsey                        \ncorrect=male     guess=female   name=Kory                          \ncorrect=male     guess=female   name=Lemmy                         \ncorrect=male     guess=female   name=Lenny                         \ncorrect=male     guess=female   name=Lesley                        \ncorrect=male     guess=female   name=Luke                          \ncorrect=male     guess=female   name=Martie                        \ncorrect=male     guess=female   name=Marty                         \ncorrect=male     guess=female   name=Max                           \ncorrect=male     guess=female   name=Mika                          \ncorrect=male     guess=female   name=Mischa                        \ncorrect=male     guess=female   name=Mitch                         \ncorrect=male     guess=female   name=Moise                         \ncorrect=male     guess=female   name=Monte                         \ncorrect=male     guess=female   name=Morly                         \ncorrect=male     guess=female   name=Morty                         \ncorrect=male     guess=female   name=Murdoch                       \ncorrect=male     guess=female   name=Mustafa                       \ncorrect=male     guess=female   name=Neale                         \ncorrect=male     guess=female   name=Neddy                         \ncorrect=male     guess=female   name=Obie                          \ncorrect=male     guess=female   name=Paddie                        \ncorrect=male     guess=female   name=Pascale                       \ncorrect=male     guess=female   name=Pepe                          \ncorrect=male     guess=female   name=Prentice                      \ncorrect=male     guess=female   name=Quigly                        \ncorrect=male     guess=female   name=Rabi                          \ncorrect=male     guess=female   name=Rafe                          \ncorrect=male     guess=female   name=Ralph                         \ncorrect=male     guess=female   name=Rawley                        \ncorrect=male     guess=female   name=Reese                         \ncorrect=male     guess=female   name=Rey                           \ncorrect=male     guess=female   name=Ronny                         \ncorrect=male     guess=female   name=Rourke                        \ncorrect=male     guess=female   name=Rudie                         \ncorrect=male     guess=female   name=Rutledge                      \ncorrect=male     guess=female   name=Say                           \ncorrect=male     guess=female   name=Shaine                        \ncorrect=male     guess=female   name=Stacy                         \ncorrect=male     guess=female   name=Stanley                       \ncorrect=male     guess=female   name=Tabbie                        \ncorrect=male     guess=female   name=Tally                         \ncorrect=male     guess=female   name=Tarrance                      \ncorrect=male     guess=female   name=Temple                        \ncorrect=male     guess=female   name=Thayne                        \ncorrect=male     guess=female   name=Thorpe                        \ncorrect=male     guess=female   name=Torey                         \ncorrect=male     guess=female   name=Torre                         \ncorrect=male     guess=female   name=Trace                         \ncorrect=male     guess=female   name=Tracey                        \ncorrect=male     guess=female   name=Udale                         \ncorrect=male     guess=female   name=Uri                           \ncorrect=male     guess=female   name=Vassily                       \ncorrect=male     guess=female   name=Vijay                         \ncorrect=male     guess=female   name=Vinnie                        \ncorrect=male     guess=female   name=Waite                         \ncorrect=male     guess=female   name=Wallache                      \ncorrect=male     guess=female   name=Waverley                      \ncorrect=male     guess=female   name=Westbrooke                    \ncorrect=male     guess=female   name=Westleigh                     \ncorrect=male     guess=female   name=Willie                        \ncorrect=male     guess=female   name=Willy                         \ncorrect=male     guess=female   name=Yancey                        \ncorrect=male     guess=female   name=Yancy                         \ncorrect=male     guess=female   name=Yehudi                        \ncorrect=male     guess=female   name=Yuri                          \ncorrect=male     guess=female   name=Zackariah                     \ncorrect=male     guess=female   name=Zechariah                     \ncorrect=male     guess=female   name=Zollie                        \n"
        }
      ]
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%% md\n"
        }
      },
      "cell_type": "markdown",
      "source": "## 10. Intégration des nouveaux features dans le classifieur\n\nIl semble bénéfique d'ajuster nos features en incluant les deux dernières lettres.\n\nEt youpi ! La précision est montée de 76% à 78.1%. C'est pas mal non ?"
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%%\n"
        },
        "trusted": false
      },
      "cell_type": "code",
      "source": "def gender_features(word):\n     return {'suffix1': word[-1:],\n             'suffix2': word[-2:]}\n\ntrain_set = [(gender_features(n), gender) for (n, gender) in train_names]\ndevtest_set = [(gender_features(n), gender) for (n, gender) in devtest_names]\nclassifier = nltk.NaiveBayesClassifier.train(train_set)\nprint(nltk.classify.accuracy(classifier, devtest_set))\n\n# 0.76 -> 0.781",
      "execution_count": 19,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "0.781\n"
        }
      ]
    },
    {
      "metadata": {
        "pycharm": {
          "name": "#%% md\n"
        }
      },
      "cell_type": "markdown",
      "source": "## 11. Importance de nouveaux splits train/dev\n\nNous pouvons donc réitérer ce processus d'analyse d'erreurs et de feature engineering jusqu'à obtenir une performance satisfaisante.\n\nAttention !:D\n\nIl vaut mieux faire un nouveau split train/dev à chaque fois qu'on intègre/supprime des features pour éviter l'overfitting.\n\n\n## Conclusions\n\nBravo d'avoir fini l'article !\n\nCe qu'il faut retenir :\n\n1. Le machine learning traditionnel repose pas mal sur l'analyse humaine. Comme vous avez vu ici, l'analyse des erreurs de classification aide beaucoup l'intelligence \"artificielle\".\n\n2. Il est important de faire un split train/dev/test pour éviter que le modèle soit overfitted. Dans la même lignée de pensée il est aussi conseillé de garder un nombre raisonnable de features.\n\n3. Vous l'aurez compris. L'analyse d'erreurs (feature engineering) et le compromis entre performance et généralisabilité font du machine learning un art qui nécessite un savoir-faire qui s'acquiert au fil des ans.\n\n[Reference](https://www.nltk.org/book/ch06.html)\n"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "myenv",
      "display_name": "Python (myenv)",
      "language": "python"
    },
    "language_info": {
      "name": "python",
      "version": "3.7.6",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "nbTranslate": {
      "hotkey": "alt-t",
      "sourceLang": "en",
      "targetLang": "fr",
      "displayLangs": [
        "*"
      ],
      "langInMainMenu": true,
      "useGoogleTranslate": true
    },
    "toc": {
      "nav_menu": {},
      "number_sections": true,
      "sideBar": false,
      "skip_h1_title": true,
      "base_numbering": 1,
      "title_cell": "Table des matières",
      "title_sidebar": "Contents",
      "toc_cell": false,
      "toc_position": {},
      "toc_section_display": true,
      "toc_window_display": false
    },
    "varInspector": {
      "window_display": false,
      "cols": {
        "lenName": 16,
        "lenType": 16,
        "lenVar": 40
      },
      "kernels_config": {
        "python": {
          "library": "var_list.py",
          "delete_cmd_prefix": "del ",
          "delete_cmd_postfix": "",
          "varRefreshCmd": "print(var_dic_list())"
        },
        "r": {
          "library": "var_list.r",
          "delete_cmd_prefix": "rm(",
          "delete_cmd_postfix": ") ",
          "varRefreshCmd": "cat(var_dic_list()) "
        }
      },
      "types_to_exclude": [
        "module",
        "function",
        "builtin_function_or_method",
        "instance",
        "_Feature"
      ]
    },
    "gist": {
      "id": "",
      "data": {
        "description": "projetTal1_classification_prenoms",
        "public": true
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 1
}