Skip to content

Instantly share code, notes, and snippets.

@pythonlessons

pythonlessons/transformers_training_11.py

Created September 4, 2023 15:04
Show Gist options
  • Save pythonlessons/5bea63388f23a6bce927d45168f8a15e to your computer and use it in GitHub Desktop.
Save pythonlessons/5bea63388f23a6bce927d45168f8a15e to your computer and use it in GitHub Desktop.
Download ZIP
transformers_training
Raw
transformers_training_11.py
optimizer = tf.keras.optimizers.Adam(learning_rate=configs.init_lr, beta_1=0.9, beta_2=0.98, epsilon=1e-9)
# Compile the model
transformer.compile(
loss=MaskedLoss(),
optimizer=optimizer,
metrics=[MaskedAccuracy()],
run_eagerly=False
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment