Ляшевская Ольга Николаевна

The application of corpus-based language distance measurement to the diatopic variation study (on the material of the Old Novgorodian birchbark letters)

2025 · CHAPTER · en

The paper presents a computer-assisted exploration of a set of texts, where qualitative analysis complements the linguistically-aware vector-based language distance measurements, interpreting them through close reading and thus proving or disproving their conclusions. It proposes using a method designed for small raw corpora to explore the individual, chronological, and gender-based differences within an extinct single territorial lect, known only by a scarce collection of documents. The material under consideration is the Novgorodian birchbark letters, a set of rather small manuscripts (not a single one is more than 1000 tokens) that are witnesses of the Old Novgorodian lect, spoken on the territories of modern Novgorod and Staraya Russa at the first half of the first millennium CE. The study shows the existence of chronological variation, a mild degree of individual variation, and almost absent gender-based differences. Possible prospects of the study include its application to the newly discovered birchbark letters and using an outgroup for more precise measurements.

BERT-like Models for Slavic Morpheme Segmentation

2025 · CHAPTER · en

Алгоритмы автоматической сегментации морфем применимы в различных задачах, таких как построение токенизаторов и обучение языку. Для славянских языков разработка таких алгоритмов осложняется богатыми деривационными возможностями этих языков. Предыдущие исследования показали, что в среднем эти алгоритмы уже достигли экспертного уровня качества. Однако ключевой нерешенной проблемой является значительное снижение производительности при сегментации слов, содержащих корни, отсутствующие в обучающих данных. Эту проблему можно частично решить, используя предварительно обученные языковые модели для лучшего учета семантики слов. В данной работе мы исследовали возможность тонкой настройки моделей типа BERT для сегментации морфем, используя данные белорусского, чешского и русского языков. Мы обнаружили, что для чешского и русского языков наши модели превосходят все ранее предложенные подходы, достигая точности на уровне слов 92,5-95,1%. Для белорусского языка эта задача была решена впервые.

Transformer-based approaches for lemmatizing abbreviations in Russian texts

2025 · ARTICLE · en

This paper addresses the task of lemmatizing abbreviations in the Russian language. Abbreviation lemmatization is particularly challenging, as it involves not only transforming a word into its normal form but also correctly expanding the abbreviation. We explore two approaches to this task, both leveraging large pretrained language models. The first approach is generative, where the lemma is produced as a textual output by the model. The second approach relies on classification models to select the most appropriate lemma for abbreviations that have multiple common expansions. The paper discusses the strengths and limitations of both approaches. The experiments are conducted on Russian texts selected from the Russian National Corpus.

Rubic2: Ensemble Model for Russian Lemmatization

2025 · CHAPTER · en

Pre-trained language models have significantly advanced natural language processing (NLP), particularly in analyzing languages with complex morphological structures. This study addresses lemmatization for the Russian language, the errors in which can critically affect the performance of information retrieval, question answering, and other tasks. We present the results of experiments on generative lemmatization using pre-trained language models. Our findings demonstrate that combining generative models with the existing solutions allows achieving performance that surpasses current results for the lemmatization of Russian. This paper also introduces Rubic2, a new ensemble approach that combines the generative BART-base model, fine-tuned on a manually annotated data set of 2.1 million tokens, with the neural model called Rubic which is currently used for morphological annotation and lemmatization in the Russian National Corpus. Extensive experiments show that Rubic2 outperforms current solutions for the lemmatization of Russian, offering superior results across various text domains and contributing to advancements in NLP applications.

Дискриминативная лемматизация сокращений в эпоху LLM

2025 · ARTICLE · ru

В работе представлено исследование эффективности дискриминативных методов лемматизации сокращений в русскоязычных текстах. В отличие от генеративных подходов, дискриминативные модели осуществляют выбор оптимальной леммы из фиксированного набора вариантов, что исключает риск генерации грамматически некорректных словоформ. Впервые для русского языка проводится комплексный анализ четырех контекстно-ориентированных подходов: (1) ранжирования на основе маскированного языкового моделирования, (2) бинарной классификации, (3) многоклассовой классификации, а также (4) обучения с использованием инструкций. Особое внимание уделено случаям контекстной неоднозначности, когда одно сокращение в пределах одного текстового фрагмента соответствует разным лемам. Результаты демонстрируют, что многоклассовая классификация с дообучением предобученных моделей достигает наивысшего качества (F-мера с макроусреднением – 97.75–99.92% в зависимости от сокращения); однако в условиях ограниченного объема обучающих данных обучение с использованием инструкций и ранжирование на основе маскированного языкового моделирования показывают перспективные результаты. Кроме того, эффективность данных подходов возрастает в случаях контекстной неоднозначности. Исследование вносит вклад в развитие методов обработки русскоязычных текстов, предлагая практические рекомендации по выбору архитектур для задач лемматизации сокращений.

Fighting Evaluation Inflation: Concentrated Datasets for Grammatical Error Correction

2024 · ARTICLE · en

Background: Grammatical error correction (GEC) systems have greatly developed over the recent decade. According to common metrics, they often reach the level of or surpass human experts. Nevertheless, they perform poorly on several kinds of errors that are effortlessly corrected by humans. Thus, reaching the resolution limit, evaluation algorithms and datasets do not allow for further enhancement of GEC systems. Purpose: To solve the problem of the resolution limit in GEC. The suggested approach is to use for evaluation concentrated datasets with a higher density of errors that are difficult for modern GEC systems to handle. Method: To test the suggested solution, we look at distant-context-sensitive errors that have been acknowledged as challenging for GEC systems. We create a concentrated dataset for English with a higher density of errors of various types, half-manually aggregating pre-annotated examples from four existing datasets and further expanding the annotation of distant-context-sensitive errors. Two GEC systems are evaluated using this dataset, including traditional scoring algorithms and a novel approach modified for longer contexts. Results: The concentrated dataset includes 1,014 examples sampled manually from FCE, CoNLL-2014, BEA-2019, and REALEC. It is annotated for types of context-sensitive errors such as pronouns, verb tense, punctuation, referential device, and linking device. GEC systems show lower scores when evaluated on the dataset with a higher density of challenging errors, compared to a random dataset with otherwise the same parameters. Conclusion: The lower scores registered on concentrated datasets confirm that they provide a way for future improvement of GEC models. The dataset can be used for further studies focusing on distant-context-sensitive GEC.

Automatic Morpheme Segmentation for Russian: Can an Algorithm Replace Experts?

2024 · ARTICLE · en

Introduction: Numerous algorithms have been proposed for the task of automatic morpheme segmentation of Russian words. Due to the differences in task formulation and datasets utilized, comparing the quality of these algorithms is challenging. It is unclear whether the errors in the models are due to the ineffectiveness of algorithms themselves or to errors and inconsistencies in the morpheme dictionaries. Thus, it remains uncertain whether any algorithm can be used to automatically expand the existing morpheme dictionaries. Purpose: To compare various existing algorithms of morpheme segmentation for the Russian language and analyze their applicability in the task of automatic augmentation of various existing morpheme dictionaries. Results: In this study, we compared several state-of-the-art machine learning algorithms using three datasets structured around different segmentation paradigms. Two experiments were carried out, each employing five-fold cross-validation. In the first experiment, we randomly partitioned the dataset into five subsets. In the second, we grouped all words sharing the same root into a single subset, excluding words that contained multiple roots. During cross-validation, models were trained on four of these subsets and evaluated on the remaining one. Across both experiments, the algorithms that relied on ensembles of convolutional neural networks consistently demonstrated the highest performance. However, we observed a notable decline in accuracy when testing on words containing unfamiliar roots. We also found that, on a randomly selected set of words, the performance of these algorithms was comparable to that of human experts. Conclusion: Our results indicate that although automatic methods have, on average, reached a quality close to expert level, the lack of semantic consideration makes it impossible to use them for automatic dictionary expansion without expert validation. The conducted research revealed that further research should be aimed at addressing the key identified issues: poor performance with unknown roots and acronyms. At the same time, when a small number of unfamiliar roots can be assumed in the test dataset, an ensemble of convolutional neural networks should be utilized. The presented results can be used in the development of morpheme-oriented tokenizers and systems for analyzing the complexity of texts.

String Similarity Measures for Evaluating the Lemmatisation in Old Church Slavonic

2024 · CHAPTER · en

Национальный корпус русского языка 2.0: новые возможности и перспективы развития

2024 · ARTICLE · ru

В статье подводятся итоги проекта фундаментальной реконструкции и модернизации платформы Национального корпуса русского языка, осуществленного в 2020-2023 годах. В фокусе статьи новые возможности, которые открываются для лингвистов и более широкой аудитории, в частности, улучшение репрезентативности имеющихся корпусов, создание новых корпусов, новая разметка, полученная с помощью применения нейросетевых моделей, новые интерфейсные решения. Более детально рассматриваются три ярких новых компонента: ресурсный – новый Корпус социальных сетей, поисковый – Панхронический корпус, объединяющий поиск по корпусам разных периодов, и аналитический – функциональный комплекс статистики и визуализации данных.

Measuring language distance based on small raw corpora

2024 в печати · CHAPTER · en

Measuring language distance is a computational linguistic task of quantitatively expressing the difference between two given languages. The paper proposes a new way to measure language distance on small raw corpora material. The method is a combination of string similarity measures and frequency-based metrics. Pairwise comparisons of the analysed material by this measure are later used to get clusterisation that is tested against the existing reliable human genetic classification. The article implements two datasets for this task: texts of East Slavic small territorial lects (traditionally called dialects) and West/South Slavic standard lects. Both datasets are often challenging for computational phylogenetic linguistic methods. Experiments show that language distance may be measured efficiently by calculating a mean of a single pool of values from the results of 1) frequency-based metrics between coinciding character 3-grams, 2) joined frequency-based metrics and string similarity measures between non-coinciding character 3-grams.