Skip to content

Instantly share code, notes, and snippets.

@aloha12345x

aloha12345x/gaiic-track3-share.md Secret

Created March 23, 2021 03:31
Show Gist options
  • Save aloha12345x/b2b81c52dd5fee7b47e3a4eb537232f3 to your computer and use it in GitHub Desktop.
Save aloha12345x/b2b81c52dd5fee7b47e3a4eb537232f3 to your computer and use it in GitHub Desktop.
Download ZIP
GAIIC Track3 Share
Raw
gaiic-track3-share.md

GAIIC Track3 “猫老板”队分享

数据处理

  • 清洗：分开连在一起的英文单词，规范标点符号
  • 增强：左右交换pair

Pretrain

  • 使用了hfl/chinese-roberta-wwm-ext和hfl/chinese-roberta-wwm-ext-large
  • 单句和pair都用
  • 使用各种mask

Finetune

  • 左右交换pair，增强数据
  • hfl/chinese-roberta-wwm-ext-large：lr=1e-5，weigt clip=1
  • 五折交叉验证
  • 多个checkpoint ensemble
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment