inctf_findme.md

Misc: Find Me If You Can

First look

We are provided an address and port. Running netcat on these tells us we need to find the number not matching (we have 3 attempts). We are then given a blob of base64 encoded data. Decoding this reveals a header of 78 9c, meaning this is zlib compressed data. Decompressing gives us data with a JPEG file header, and viewing this image results in the following:

[Insert Image Here]

Approach

This is clearly a machine learning problem. In fact, the image consists of 20x20 digits, all having a size of 28x28. This is coincidentally the same size as the images in the MNIST dataset. In order to indentify these digits, we train a simple Convolutional Neural Network using the MNIST dataset for training data. We used Keras for its ease of use.

Note on classification

One noteworthy point is that the output of our CNN is an array of length 10 containing the probabilities of the image being each number. When identifying an image, the highest probability is usually used. When using a CNN with 99% accuracy, this should still lead to 4 or more candidates, where we are only allowed 3. We can get around this by realising that because we know there are 399 correct images and only 1 wrong, we can instead use the geometrical distance from the probability array of each image to the average probability array, to get a better guess of the outlier numbers (the numbers most likely to be different from all others).

Final Code

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Convolution2D, MaxPooling2D, Flatten
from pwn import *
from base64 import b64decode
from zlib import decompress
from io import BytesIO
from PIL import Image
import numpy as np

model = Sequential()
r = remote("34.70.233.147", 7777)

batch_size = 32
num_classes = 10
epochs = 10

def train():
    (x_train, y_train), (x_test, y_test) = mnist.load_data()

    x_train = x_train.reshape(x_train.shape[0], 1, 28, 28)
    x_test = x_test.reshape(x_test.shape[0], 1, 28, 28)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train /= 255
    x_test /= 255
    print(x_train.shape[0], 'train samples')
    print(x_test.shape[0], 'test samples')
    
    y_train = keras.utils.to_categorical(y_train, num_classes)
    y_test = keras.utils.to_categorical(y_test, num_classes)

    model.add(Convolution2D(32,(3,3),activation='relu',data_format='channels_first',input_shape=(1,28,28)))
    model.add(Convolution2D(32, 3, 3, activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(Dropout(0.25))
    model.add(Flatten())
    model.add(Dense(256, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(10, activation='softmax'))

    model.summary()

    model.compile(loss='categorical_crossentropy',
                  optimizer='adam',
                  metrics=['accuracy'])

    history = model.fit(x_train, y_train,
                        batch_size=batch_size,
                        epochs=epochs,
                        verbose=1,
                        validation_data=(x_test, y_test))
    score = model.evaluate(x_test, y_test, verbose=0)
    print('Test loss:', score[0])
    print('Test accuracy:', score[1])

def recvwait():
        s = b""
        temp = r.recv(timeout=3)
        while temp:
                s += temp
                try:
                    temp = r.recv(timeout=3)
                except:
                    break
        return s


def receive():
    s = recvwait().decode()
    print(s)
    s = s.split("b'")[1].strip("'")
    d = decompress(b64decode(s.encode()))
    s = BytesIO(d)
    im = Image.open(s).convert("RGB")
    #im.show()
    s.close()
    out = []
    for i in range(20):
            for j in range(20):
                    pix = np.array([[im.getpixel((28*j+dj, 28*i+di))[0] for dj in range(28)] for di in range(28)])
                    pix = pix.reshape(1,1,28,28)
                    pix = pix.astype('float32')
                    pix /= 255
                    coord = (i, j)
                    out.append((coord, pix))
    return out


train()

while True:
    imgs = receive()
    out = []
    grid = [[0 for j in range(20)] for i in range(20)]
    for img in imgs:
        res = model.predict(img[1])
        val = np.argmax(res, axis=1)[0]
        out.append((img[0], list(res[0])))
        grid[img[0][0]][img[0][1]] = val
    avg = [sum(ent[1][j] for ent in out)/len(out) for j in range(10)]
    fin = []
    for ent in out:
        dists = sum(pow(avg[i]-ent[1][i], 2) for i in range(10))
        fin.append((dists, ent[0]))
    fin = "("+", ".join(", ".join(str(i) for i in e[1]) for e in sorted(fin)[::-1][:3])+")"
    r.sendline(fin)

Running this gives the flag, inctf{1_D4Y_R0b0tS_will_rul3_th3_w0rld_4_SUR3!!}