Skip to content

Instantly share code, notes, and snippets.

@zodman
Created January 10, 2020 07:27
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save zodman/f78ef3200fafa49b17923343b6b623ab to your computer and use it in GitHub Desktop.
title tags description
my IA Challenge
django, keras
View the slide with "Slide Mode".

Andres Bernardo Vargas Rodriguez

github.com/zodman

  • I'm Python lover 🐍
  • Fullstack developer
  • Linux user expert 🐧

I want to said. My expertise in IA, machine learning, deep learning, and neurons is very rusty.

πŸ‘΄


What data.json contains?

text and topics

vagrant@ubuntu-xenial:/vagrant$ cat data.json | tail -n 2 | jq
{
  "count": 0,
  "text": "lol",
  "topics": [
    {
      "topic": 0,
      "relevance": 32,
      "confidence": 224
    }
  ],
  "player": "4446",
  "flags": 4247631,
  "client_id": 999,
  "filtered": 0,
  "simplified": "lol"
}

Keras


My plans

Text Classification πŸ‘Š

  • Using keras

Web development 🐍

  • Using django

Text Classification πŸ‘Š

  • Obtain train data and test data.
  • Define the best Keras Model for the case.
  • Pass the train data to a Keras model compiled and then evaluate the test data.
  • Check if train data works as expected

First, try

model = Sequential()
model.add(Dense(512, input_dim=vocab_size))
model.add(Activation('relu'))
  
model.add(Dense(9))
model.add(Activation('softmax'))


model.compile(loss='categorical_crossentropy', 
              optimizer='rmsprop', 
              metrics=['accuracy'])
            
Test score: 2.1609163373932803
Test accuracy: 38.34586441516876
            


Keras.Sequential Keras.Dense model.compile


Second try:

model = Sequential()
model.add(Dense(10, input_dim=vocab_size, activation="relu"))
model.add(Dense(len(num_tops), activation="sigmoid"))


## Compile the models
model.compile(loss='binary_crossentropy', 
              optimizer='adam', 
              metrics=['accuracy'])
Test score: 0.14842960193641203
Test accuracy: 96.06708884239197

πŸ‘


check training =====
###
text: her asdvice
Actual topics: [0]
Predicted topics: [0] **YEAH!*
###
text: so they took lyk 3 pics
Actual topics: [0, 3, 4]
Predicted topics: [0] **YEAH!*
###
text: ohhhh
Actual topics: [0]
Predicted topics: [0] **YEAH!*
###
text: so i was like ARGHHH
Actual topics: [0]
Predicted topics: [0] **YEAH!*
###
text: dnt laf ab god
Actual topics: [0, 3, 11, 12]
Predicted topics: [0] **YEAH!*

πŸ‘ πŸ‘ πŸ‘ πŸ‘ πŸ‘ πŸ‘


Text classification βœ…

Next ?

let's do some Web development.


With:

  • Django
  • Celery
  • django-restframework
  • vuejs (parceljs)

Simple SPA for send new text and expect to use my new script to classify new text.


Demo time.


My Conclusion

  • This challenge was FUN ❗
  • With the better understanding about the models and compile options you can obtain a better results.
  • I missing the unittest βœ‹
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment