Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
# Initialize helpers
data_container = DataHandler()
maskHandler = MaskHandler()
# Initialize parameters
num_layers = 4
num_neurons = 128
num_hidden_layers = 512
num_heads = 8
# Initialize vocabular size
input_vocablar_size = data_container.tokenizer_ru.vocab_size + 2
target_vocablar_size = data_container.tokenizer_en.vocab_size + 2
# Initialize learning rate
learning_rate = Schedule(num_neurons)
optimizer = Adam(learning_rate, beta_1=0.9, beta_2=0.98, epsilon=1e-9)
# Initialize transformer
transformer = Transformer(num_layers, num_neurons, num_hidden_layers, num_heads, input_vocablar_size, target_vocablar_size)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment