This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def triplet_loss(y_true, y_pred, alpha = ALPHA): | |
""" | |
Implementation of the triplet loss function | |
Arguments: | |
y_true -- true labels, required when you define a loss in Keras, you don't need it in this function. | |
y_pred -- python list containing three objects: | |
anchor -- the encodings for the anchor data | |
positive -- the encodings for the positive data (similar to anchor) | |
negative -- the encodings for the negative data (different from anchor) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def create_base_network(in_dims, out_dims): | |
""" | |
Base network to be shared. | |
""" | |
model = Sequential() | |
model.add(BatchNormalization(input_shape=in_dims)) | |
model.add(LSTM(512, return_sequences=True, dropout=0.2, recurrent_dropout=0.2, implementation=2)) | |
model.add(LSTM(512, return_sequences=False, dropout=0.2, recurrent_dropout=0.2, implementation=2)) | |
model.add(BatchNormalization()) | |
model.add(Dense(512, activation='relu')) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
in_dims = (N_MINS, n_feat) | |
out_dims = N_FACTORS | |
# Network definition | |
with tf.device(tf_device): | |
# Create the 3 inputs | |
anchor_in = Input(shape=in_dims) | |
pos_in = Input(shape=in_dims) | |
neg_in = Input(shape=in_dims) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Training the model | |
model.fit(train_data, y_dummie, batch_size=256, epochs=10) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def traffic_to_encoding(x, model): | |
return model.predict(np.array([x])) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def identify_traffic(x, database, model): | |
""" | |
Implements traffic recognition. | |
Arguments: | |
x -- the traffic to identify | |
database -- database containing recognized traffic encodings | |
model -- the encoding model | |
Returns: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
database = {} | |
database['normal'] = traffic_to_encoding(get_example_label(train_cases_df, df_lens, 0), base_network) | |
database['error2'] = traffic_to_encoding(get_example_label(train_cases_df, df_lens, 1), base_network) | |
# Prediction on traffic | |
identify_traffic(x, database, base_network) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
X = pmi(df) | |
X = normalize(X, copy=False) | |
tree = BallTree(X, p=2) | |
knn_d, knn_ix = tree.query([X[<specific_example>]], k=10, return_distance=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.cluster import MiniBatchKMeans | |
num_clusters = 80 | |
kn = MiniBatchKMeans(n_clusters=num_clusters, init_size=max(300, 3*num_clusters)).fit(X) | |
labels = kn.labels_ | |
n_clusters_ = len(set(labels)) - (1 if -1 in labels else 0) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis | |
clf1 = LinearDiscriminantAnalysis(n_components=2) | |
X1 = clf1.fit_transform(X, labels) |
OlderNewer