【文献阅读】Pre-Training With Whole Word Masking for Chinese BERT
CuiY,CheW,LiuT,etal.Pre-trainingwithwholewordmaskingforchinesebert[J].IEEE/ACMTransactionsonAudio,Speech,andLanguageProcessing,2021,29:3504-3514.Abstract在本文中,我们旨在首先介绍中文BERT的全词掩蔽(wwm)策略,以及一系列中文预训练语言模型。