2026-10-13T15:24:25Z https://www.scidb.cn/oai

10.57760/sciencedb.28967 2025-10-09T17:59:43Z

2025-10-09 RoleEval doi:10.57760/sciencedb.28967 en RoleEval is a bilingual benchmark designed to assess the memorization, utilization, and reasoning capabilities of role knowledge for large language models. RoleEval comprises RoleEval-Global (including internationally recognized characters) and RoleEval-Chinese (including characters popular in China), with 6,000 Chinese-English parallel multiple-choice questions focusing on 300 influential people and fictional characters drawn from a variety of domains including celebrities, anime, comics, movies, TV series, games, and fiction. These questions cover basic knowledge and multi-hop reasoning abilities, aiming to systematically probe various aspects such as personal information, relationships, abilities, and experiences of the characters. To maintain high standards, we perform a hybrid quality check process combining automatic and human verification, ensuring that the questions are diverse, challenging, and discriminative. Role-playing Evaluation; Large Language Models; Bilingual Benchmark Tianhao Shen PUBLIC https://creativecommons.org/licenses/by-nc-sa/4.0/ dataset Science Data Bank