Towards Debiasing Sentence Representations (Liang et al., 2020) English BERT and ELMO
Measuring Bias in Contextualized Word Representations (Kurita et al., 2019) analyzes English BERT with word-association https://aclweb.org/anthology/W19-3823/
Semantics derived automatically from language corpora contain human-like biases (Caliskan, Bryson, and Narayanan, 2017) https://arxiv.org/abs/1608.07187
Wikipedia is a mirror of the world's gender biases (Wikimedia Fdn, 2018) wikimediafoundation.org/news/2018/10/18/wikipedia-mirror-world-gender-biases/
A larger-scale program could use Google's GAP dataset. There are several open source projects which counteract bias in BERT, ELMo, or GloVe embeddings using this Gendered Pronoun Resolution dataset. Many were created in response to a Kaggle competition in 2019. Wikipedia's https://wiki-gender.github.io - "exploring gender linguistic bias in the overview of Wikipedia biographies."
Gender bias, especially when reduced to this binary level, is a small piece of the puzzle. Consider this research from Dr. Vinodkumar Prabhakaran about biases in language models towards people with disabilities - the language is not as easily detectable or 'flippable'. https://twitter.com/vinodkpg/status/1260023228489064448
Unintended Bias in Misogyny Detection (Nozza et al., 2020) https://dnozza.github.io/publication/2019_unintended_bias_misogyny_detection/
Resolving Gendered Ambiguous Pronouns with BERT (Ionita et al., 2019)
- https://arxiv.org/abs/1906.01161
- https://github.com/Yorko/gender-unbiased_BERT-based_pronoun_resolution
Gendered Pronoun Resolution using BERT and an extractive question answering formulation (Chada 2019)
Debiasing Word Embeddings Improves Multimodal Machine Translation (Hirasawa and Komachi, 2019)