Neural Network for Assessing English Language Proficiency Developed at HSE University
The AI Lingua Neural Network has been collaboratively developed by the HSE University’s AI Research Centre, School of Foreign Languages, and online campus. The model has been trained on thousands of expert assessments of both oral and written texts. The system evaluates an individual's ability to communicate in English verbally and in writing.
Individuals interested in assessing their speaking and writing proficiency are invited to complete a series of tasks, including writing an essay, answering interview questions, and composing a monologue. Completing the tasks takes no more than 60 minutes, and the test is accessible on any digital device. Results, along with brief comments, are available within 30 minutes.
This format enables a representative assessment of the range of skills most commonly used in oral and written communication. When evaluating responses, the neural network considers over 45 parameters developed by experts at the HSE School of Foreign Languages. The result is determined through a comprehensive assessment of lexical, grammatical, and phonetic/stylistic aspects of speech, as well as characteristics such as coherence, logical presentation of thoughts, and skills in description, reasoning, and narration.
Based on the results of the tasks, a final score is calculated, and a language proficiency level is assigned, ranging from elementary to upper-intermediate. The proficiency level scale developed at the HSE School of Foreign Languages and integrated into the neural network aligns with all international language testing systems.
'An AI model capable of evaluating oral and written tasks with open-ended responses remains a rarity in language education. It can be described as a cutting-edge category of AI-based assessment systems, as the neural network has been trained to identify a variety of complex indicators of speech behaviour to determine candidates' levels of communicative competence,' according to Ekaterina Kolesnikova, Head of the HSE School of Foreign Languages.
The model has been trained on thousands of expert assessments of oral and written texts produced by actual participants in academic communication. The process of ongoing training and refinement of the model continues. Currently, the neural network is in its beta stage, undergoing testing and calibration for further integration into the process of assessing student work as part of independent English proficiency examination. It is planned for the neural network to function as one of the experts.
'The project to develop AI Lingua is an example of adapting large foundational models for a specific applied field—education. We have come all the way from testing various scientific hypotheses to implementing models in the final product, which is available as an online service for self-assessing English proficiency. Our solution is distinguished by its ability to simultaneously evaluate not only text responses but also audio speech, producing a language proficiency level based on a comprehensive set of criteria. To achieve this, we studied a substantial number of open-source solutions and subsequently conducted additional training on works analysed by professional linguists,' explains Alexey Masyutin, Director of the AI Centre.
The HSE University's team also notes that there are plans to develop models for other foreign languages. This will assist foreign language learners in conducting self-assessments of their knowledge, receiving timely feedback with the necessary level of detail, and tracking their progress in language proficiency development.
'The integration of AI solutions into the language skills assessment process fundamentally transforms the approach to language learning, making it more convenient and user-friendly. We are developing a tool that not only records the current level of knowledge but also dynamically tracks progress, offering everyone the opportunity for regular and objective self-assessment,' emphasises Sergey Roshchin, Vice Rector of HSE University.
The beta version of the English proficiency self-assessment tool is available for free with a promo code sent via email upon registration.
See also:
HSE’S Achievements in AI Presented at AIJ
The AI Journey international conference hosted a session led by Deputy Prime Minister Dmitry Chernyshenko highlighting the achievements of Russian research centres in artificial intelligence. Alexey Masyutin, Head of the HSE AI Research Centre, showcased the centre’s key developments.
Drivers of Progress and Sources of Revenue: The Role of Universities in Technology Transfer
In the modern world, the effective transfer of socio-economic and humanities-based knowledge to the real economy and public administration is essential. Universities play a decisive role in this process. They have the capability to unite diverse teams and, in partnership with the state and businesses, develop and enhance advanced technologies.
AI on Guard of Ecology: Students from Russia and Belarus Propose New Solutions to Environmental Problems
An international online hackathon dedicated to solving environmental problems was held at HSE University in Nizhny Novgorod. Students employed artificial intelligence and computer vision to develop innovative solutions for image segmentation, predictive modelling (forecasting future events based on data from the past) of emissions and creating chatbots for nature reserves and national parks.
Taming the Element: How AI Is Integrating into the Educational Process Around the World
Artificial intelligence is gradually becoming an indispensable part of higher education. Both students and teachers use it to reduce the volume of routine tasks and expand their capabilities. The limitations and prospects of AI are discussed in the report ‘The Beginning of the End or a New Era? The Effects of Generative Artificial Intelligence (GAI) in Higher Education,’ published in the journal Modern Education Analytics, under the scientific supervision of HSE Academic Supervisor Yaroslav Kuzminov.
A New Tool Designed to Assess AI Ethics in Medicine Developed at HSE University
A team of researchers at the HSE AI Research Centre has created an index to evaluate the ethical standards of artificial intelligence (AI) systems used in medicine. This tool is designed to minimise potential risks and promote safer development and implementation of AI technologies in medical practice.
HSE Researchers Develop Novel Approach to Evaluating AI Applications in Education
Researchers at HSE University have proposed a novel approach to assessing AI's competency in educational settings. The approach is grounded in psychometric principles and has been empirically tested using the GPT-4 model. This marks the first step in evaluating the true readiness of generative models to serve as assistants for teachers or students. The results have been published in arXiv.
‘Philosophy Is Thinking Outside the Box’
In October 2024, Louis Vervoort, Associate Professor at the School of Philosophy and Cultural Studies of the Faculty of Humanities presented his report ‘Gettier's Problem and Quine's Epistemic Holism: A Unified Account’ at the Formal Philosophy seminar, which covered one of the basic problems of contemporary epistemology. What are the limitations of physics as a science? What are the dangers of AI? How to survive the Russian cold? Louis Vervoort discussed these and many other questions in his interview with the HSE News Service.
HSE Scientists Propose AI-Driven Solutions for Medical Applications
Artificial intelligence will not replace medical professionals but can serve as an excellent assistant to them. Healthcare requires advanced technologies capable of rapidly analysing and monitoring patients' conditions. HSE scientists have integrated AI in preoperative planning and postoperative outcome evaluation for spinal surgery and developed an automated intelligent system to assess the biomechanics of the arms and legs.
HSE University and Sber Researchers to Make AI More Empathetic
Researchers at the HSE AI Research Centre and Sber AI Lab have developed a special system that, using large language models, will make artificial intelligence (AI) more emotional when communicating with a person. Multi-agent models, which are gaining popularity, will be engaged in the synthesis of AI emotions. The article on this conducted research was published as part of the International Joint Conference on Artificial Intelligence (IJCAI) 2024.
HSE University and Yandex to Host International AI Olympiad for Students
The HSE Faculty of Computer Science and Yandex Education are launching their first joint AI competition, Artificial Intelligence and Data Analysis Olympiad (AIDAO), for students from around the world. Participants will tackle challenging tasks in science and industry and interact with experts from HSE and Yandex. The winners will receive cash prizes.