![]() ![]() īowman, S., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. The data and relevant code can be found at. We describe the creation and characteristics of BLUEX and establish a benchmark through experiments with state-of-the-art LMs, demonstrating its potential for advancing the state-of-the-art in natural language understanding and reasoning in Portuguese. The dataset is also annotated to indicate the position of images in each question, providing a valuable resource for advancing the state-of-the-art in multimodal language understanding and reasoning. Furthermore, BLUEX includes a collection of recently administered exams that are unlikely to be included in the training data of many popular LMs as of 2023. ![]() The dataset includes annotated metadata for evaluating the performance of NLP models on a variety of subjects. To address this gap, we introduce the Brazilian Leading Universities Entrance eXams (BLUEX), a dataset of entrance exams from the two leading universities in Brazil: UNICAMP and USP. This is mainly due to the lack of high-quality datasets available to the community for carrying out evaluations in Portuguese. However, despite being the fifth most spoken language worldwide, few such evaluations have been conducted in Portuguese. ![]() One common trend in recent studies of language models (LMs) is the use of standardized tests for evaluation. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |