2026-10-12T06:14:00Z https://www.scidb.cn/oai

10.11922/sciencedb.j00001.00340 2022-01-28T14:59:01Z

2022-01-28 Evaluation data set of Word Segmentation technology in Minority Languages (MLWS2021) doi:10.11922/sciencedb.j00001.00340 en MLWS2021 word segmentation evaluation data set includes Mongolian, Tibetan and Uyghur languages. The evaluation object is the core technology of automatic word segmentation in Mongolian, Uyghur and Tibetan Languages. On the basis of MLWS2017, the data set is expanded from the previous news field to news, economy, law, entertainment and other fields; The data scale has also expanded from more than 30000 sentences before to 155000 sentences at present. The dataset includes three data files, including: (1) Tibetan.Zip is Tibetan word segmentation and annotation data, with a data volume of 25000 sentences and a file size of 1.52MB; (2) Mongolian.Zip is Mongolian word segmentation and annotation data, with a data volume of 65000 sentences and a file size of 3.16MB; (3)Uyghur.Zip is Uighur word segmentation and annotation data, with a data volume of 65000 sentences and a file size of 5.12MB.  minority language; word segmentation; evaluation dataset; standard specification for word segmentation Zhao Xiaobing Gao Lu Gao Dingguo Bao Wugedele Mieradilijiang Maimaiti Liu Yang Cai Zhijie Sun Yuan PUBLIC https://creativecommons.org/licenses/by/4.0/ dataset http://www.doi.org/10.11922/11-6035.csd.2021.0091.zh Science Data Bank