CMMLU: Measuring massive multitask language understanding in Chinese Homepage: https://github.com/haonan li/CMMLU Repository: https://huggingface.co/datasets/haonan li/cmmlu Paper: CMMLU: Measuring Chinese Massive Multitask Language Understanding. Table of Contents Introduction Leaderboard Data Citation License Introduction CMMLU is a comprehensive Chinese assessment suite specifically designed to evaluate the advanced knowledge and reasoning abilities of LLMs within the Chinese language and cultural context. CMMLU covers a wide range of subjects, comprising 67 topics that span from elementary to advanced professional levels. It includes subjects that require computational expertise, such as physics and mathematics, as well as disciplines within humanities and social sciences. Many of these tasks are not easily translatable from other languages due to their specific contextual nuances and wording. Furthermore, numerous tasks within CMMLU have answers that are specific to China and may not be universally applicable or considered correct in other regions or languages. Leaderboard Latest leaderboard is in our github. Data We provide development and test dataset for each of 67 subjects…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy