Макаров Илья Андреевич

CA-SER: Cross-Attention Feature Fusion for Speech Emotion Recognition

2024 · CHAPTER · en

In this paper, we introduce a novel tool for speech emotion recognition, CA-SER, that borrows self-supervised learning to extract semantic speech representations from a pre-trained wav2vec 2.0 model and combine them with spectral audio features to improve speech emotion recognition. Our approach involves a self-attention encoder on MFCC features to capture meaningful patterns in audio sequences. These MFCC features are combined with high-level representations using a multi-head cross-attention mechanism. Evaluation of speech emotion recognition on the IEMOCAP dataset shows that our system achieves a weighted accuracy of 74.6%, outperforming most existing techniques.

Plug-and-play unsupervised fault detection and diagnosis for complex industrial monitoring

2024 · CHAPTER · en

Today industrial facilities are equipped with lots of sensors throughout all the production line for monitoring means. Gathered data can be used to detect and predict failures; however, manual labeling of large amounts of data for supervised learning is complicated. This paper introduces an innovative approach to unsupervised fault detection and diagnosis tailored for monitoring industrial chemical processes. We showcase the efficacy of our model using two publicly accessible datasets from the Tennessee Eastman Process, each containing various faults. Furthermore, we illustrate that by fine-tuning the model on a limited amount of labeled data, it achieves performance close to that of a state-of-the-art model trained on the entire dataset.

VIA AI: Reliable Deep Reinforcement Learning for Traffic Signal Control

2024 · CHAPTER · en

Traffic signal control optimization is an integral part of any modern transportation system. However, modern traffic signal control systems often rely on predetermined fixed rules to adjust traffic signal timings. This paper presents VIA AI - an intelligent traffic signal control system that leverages deep reinforcement learning (RL) applied to count-based traffic data. Our solution offers additional adaptability and flexibility by allowing the system to learn and adjust its strategies based on real-time feedback and environmental changes. We test our approach using real-world traffic data and show that it outperforms classical methods of intersection control.

Benchmarking and Data Synthesis for Colorization of Manga Sequential Pages for Augmented Reality

2024 · CHAPTER · en

This paper introduces an innovative approach to manga colorization within augmented reality (AR) environments, focusing on the unique challenges posed by colorizing photos of manga books. We present a novel method using diffusion models to generate a synthetic dataset that accurately replicates photographed manga pages. Additionally, we have compiled a dataset of real manga photographs, capturing diverse environmental conditions. Integrating these datasets, we established a comprehensive benchmark to evaluate colorization models in scenarios that simulate AR applications. This benchmark was validated through a human study, confirming the accuracy of our metrics across both datasets. We also showed that domain adaptation may improve model performance. Paving the way for practical applications, our framework enables the creation of an AR application designed to execute manga colorization effectively.

Do You Remember the Future? Weak-to-Strong Generalization in 3D Object Detection

2024 · CHAPTER · en

This paper demonstrates a novel method for LiDAR-based 3D object detection, addressing major field challenges: sparsity and occlusion. Our approach leverages temporal point cloud sequences to generate frames that provide comprehensive views of objects from multiple angles. To address the challenge of generating these frames in real-time, we employ Knowledge Distillation within a Teacher-Student framework, allowing the Student model to emulate the Teacher's advanced perception. We pioneered the application of weak-to-strong generalization in computer vision by training our Teacher model on enriched, object-complete data. In this demo, we showcase the exceptional quality of labels produced by the X-Ray Teacher on object-complete frames, showing our method distilling its knowledge to enhance object 3D detection models.

SwiftDepth++: An Efficient and Lightweight Model for Accurate Depth Estimation.

2024 в печати · ARTICLE · en

Depth estimation is a crucial task across various domains, but the high cost of collecting labeled depth data has led to growing interest in self-supervised monocular depth estimation methods. In this paper, we introduce SwiftDepth++, a lightweight depth estimation model that delivers competitive results while maintaining a low computational budget. The core innovation of SwiftDepth++ lies in its novel depth decoder, which enhances efficiency by rapidly compressing features while preserving essential information. Additionally, we incorporate a teacher-student knowledge distillation framework that guides the student model in refining its predictions.

GEEF: A neural network model for automatic essay feedback generation by integrating writing skills assessment

2024 · ARTICLE · en

Оценка способности студентов к написанию сочинений всегда была важной составляющей исследований в области автоматической оценки эссе. Существующие методы обычно сосредотачиваются на выдаче численных оценок, и мало исследований, касающихся предоставления обратной связи о качестве письма на основе техник генерации текста. Мы считаем, что генерация качественной обратной связи при написании эссе является более ценной в качестве справочного материала. В этом исследовании мы решаем проблему генерации обратной связи по эссе, предлагая модель нейронной сети с кодировщиком-декодером под названием GEEF (Generate Essay Feedback), и предполагаем, что обратная связь пишется на основе текста исходного эссе и оценки важных навыков письма. Помимо текста входного эссе, наша модель также учитывает дополнительные характеристики, включая плавность, связность, богатство и литературный талант. Мы создаем корпус обратной связи по эссе вместе с ресурсами, помеченными человеком, для облегчения исследования. Экспериментальные результаты демонстрируют, что предложенный подход показывает многообещающую эффективность по сравнению с другими базовыми методами на автоматических и человеческих метриках оценки. Сгенерированные предложения обратной связи по различным аспектам написания эссе могут помочь студентам понять их сильные и слабые стороны в навыках письма. Кроме того, потенциальное применение этого исследования заключается в том, что оно может помочь экспертам писать более сложную обратную связь на этой основе, так как часто бывает трудно запомнить многие общепринятые шаблоны обратной связи при написании комментариев.

Refining the ONCE Benchmark With Hyperparameter Tuning

2024 · ARTICLE · en

In response to the growing demand for 3D object detection in applications such as autonomous driving, robotics, and augmented reality, this work focuses on the evaluation of semi-supervised learning approaches for point cloud data. The point cloud representation provides reliable and consistent observations regardless of lighting conditions, thanks to advances in LiDAR sensors. Data annotation is of paramount importance in the context of LiDAR applications, and automating 3D data annotation with semi-supervised methods is a pivotal challenge that promises to reduce the associated workload and facilitate the emergence of cost-effective LiDAR solutions. Nevertheless, the task of semi-supervised learning in the context of unordered point cloud data remains formidable due to the inherent sparsity and incomplete shapes that hinder the generation of accurate pseudo-labels. In this study, we consider these challenges by posing the question: “To what extent does unlabelled data contribute to the enhancement of model performance?” We show that improvements from previous semi-supervised methods may not be as profound as previously thought. Our results suggest that simple grid search hyperparameter tuning applied to a supervised model can lead to state-of-the-art performance on the ONCE dataset, while the contribution of unlabelled data appears to be comparatively less exceptional.

Adversarial Attacks and Defenses in Fault Detection and Diagnosis: A Comprehensive Benchmark on the Tennessee Eastman Process

2024 в печати · ARTICLE · en

Integrating machine learning into Automated Control Systems (ACS) enhances decision-making in industrial process management. One of the limitations to the widespread adoption of these technologies in industry is the vulnerability of neural networks to adversarial attacks. This study explores the threats in deploying deep learning models for Fault Detection and Diagnosis (FDD) in ACS using the Tennessee Eastman Process dataset. By evaluating three neural networks with different architectures, we subject them to six types of adversarial attacks and explore five different defense methods. Our results highlight the strong vulnerability of models to adversarial samples and the varying effectiveness of defense strategies. We also propose a new defense strategy based on combining adversarial training and data quantization. This research contributes several insights into securing machine learning within ACS, ensuring robust FDD in industrial processes.

Weak-to-Strong 3D Object Detection with X-Ray Distillation

2024 · CHAPTER · en