Макаров Илья Андреевич

Robust Manga Page Colorization via Coloring Latent Space

2023 · ARTICLE · en

Manga (Japanese comics) are commonly drawn with black ink on paper. Colorization of manga pages can enrich the visual content and provide a better reading experience. However, the existing colorization approaches are not sufficiently robust. In this paper, we propose a two-stage approach for manga page colorization that supports sampling and color modification with color hints. In the first step, we employ the Pixel2Style2Pixel architecture to map the black-and-white manga image into the latent space of StyleGAN pretrained on the highly blurred colored manga images that we call Coloring Latent Space. The latent vector is automatically or manually modified and fed into the StyleGAN synthesis network to generate a coloring draft that sets the overall color distribution for the image. In the second step, heavy Pix2Pix-like conditional GAN fuses the information from the coloring draft and user-defined color hints and generates the final high-quality coloring. Our method partially overcomes the multimodality of the considered problem and generates diverse but consistent colorings without user input. The visual comparison, the quantitative evaluation with Frechet Inception Distance, and the qualitative evaluation via Mean Opinion Score exhibit the superiority of our approach over the existing state-of-the-art manga pages colorization method.

Comparative Analysis of Logic Reasoning and Graph Neural Networks for Ontology-Mediated Query Answering with a Covering Axiom

2023 · ARTICLE · en

The problem of query answering over incomplete attributed graph data is a challenging field of database management systems and artificial intelligence. When there are rules on data structure expressed in the form of the ontology, the theoretical complexity of finding exact solution satisfying ontology constraints increases. Logic-based methods use theoretical constructions to obtain efficient rewritings of the original queries with respect to ontology and find an answer to the rewriting query over incomplete data. However, there is an opportunity to use faster machine learning methods to label all the data and query over the ‘‘most probable’’ data model without taking into account the ontology. This research paper investigates the effectiveness and trustworthiness of both mentioned approaches for answering ontology-mediated queries on graph databases that integrate an ontology with a covering axiom, which states that every node belongs to either of two classes. The first approach involves finding precise answers through logical reasoning and rewriting the problem into a datalog program, while the second approach employs a trained graph neural network to label data in a binary classification problem and leverages SQL for query answering. We conduct an in-depth analysis of the time performance of these approaches and evaluate the impact of training set selection on their ability of correct query answering. By comparing these approaches across various experiments, we provide insights into their strengths and limitations for answering ontology-mediated queries containing a Boolean conjunctive query. In particular, we showed the importance of logic-based approaches for ontology with a covering axiom and the inability of machine learning methods to find answers for ontology-mediated queries in large networks.

Interaction models for remaining useful lifetime estimation

2023 · ARTICLE · en

Статья посвящена проблеме контроля состояния промышленных устройств по показаниям их датчиков. Существующие методы основаны на подходе к извлечению признаков, в котором происходит предсказание. Мы предлагаем метод взаимодействия нескольких блоков различной сложности, которые по-разному агрегируют информацию во времени, для создания общего скрытого пространства для предсказания оставшегося срока службы (RUL), и обучаем полученную архитектуру за один проход с новой функцией потерь, направленной на гетерогенное скрытое пространство. Новая модель TFI, основанная на агрегации информации с учетом показаний датчиков и адаптированной иерархической свертке, показала лучшие результаты на наборе данных C-MAPSS.

Temporal network embedding framework with causal anonymous walks representations

2022 · ARTICLE · en

Research Papers Recommendation

2022 · CHAPTER · en

The work is devoted to academic papers recommendation task considered as link prediction on a static citation network. We compare several graph embeddings, text-based and fusion models in the link prediction problem on academic papers citation dataset. We showed that fusion models of graph and text information outperform other approaches based on graph or text information alone. We prove this via an extensive set of experiments with different train/test splits that our fusion models are robust and retain superior performance even with a reduced train set.

Temporal network embedding framework with causal anonymous walks representations

2022 · ARTICLE · en

Many tasks in graph machine learning, such as link prediction and node classification, are typically solved using representation learning. Each node or edge in the network is encoded via an embedding. Though there exists a lot of network embeddings for static graphs, the task becomes much more complicated when the dynamic (i.e., temporal) network is analyzed. In this paper, we propose a novel approach for dynamic network representation learning based on Temporal Graph Network by using a highly custom message generating function by extracting Causal Anonymous Walks. We provide a benchmark pipeline for the evaluation of temporal network embeddings. This work provides the first comprehensive comparison framework for temporal network representation learning for graph machine learning problems involving node classification and link prediction in every available setting. The proposed model outperforms state-of-the-art baseline models. The work also justifies their difference based on evaluation in various transductive/inductive edge/node classification tasks. In addition, we show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks, involving credit scoring based on transaction data.

Self-supervised recurrent depth estimation with attention mechanisms

2022 · ARTICLE · en

Depth estimation has been an essential task for many computer vision applications, especially in autonomous driving, where safety is paramount. Depth can be estimated not only with traditional supervised learning but also via a self-supervised approach that relies on camera motion and does not require ground truth depth maps. Recently, major improvements have been introduced to make self-supervised depth prediction more precise. However, most existing approaches still focus on single-frame depth estimation, even in the self-supervised setting. Since most methods can operate with frame sequences, we believe that the quality of current models can be significantly improved with the help of information about previous frames. In this work, we study different ways of integrating recurrent blocks and attention mechanisms into a common self-supervised depth estimation pipeline. We propose a set of modifications that utilize temporal information from previous frames and provide new neural network architectures for monocular depth estimation in a self-supervised manner. Our experiments on the KITTI dataset show that proposed modifications can be an effective tool for exploiting temporal information in a depth prediction pipeline.

Сontext-dependent Word Embeddings for Word Sense Induction in Russian Language

2022 · CHAPTER · en

In the present work, contextualized word embeddings such as provided by ELMo or BERT are applied to the Word Sense Induction (WSI) task for the Russian language. Since embeddings produced by these models depend on context, we presumed that they could be able to capture the particular word meaning used in a particular sentence. We have tested it on the three datasets available for Russian language WSI task. We created a WSI system for Russian language based on clustering context-dependent word embeddings constructed by pre-trained language models.

Survey on graph embeddings and their applications to machine learning problems on graphs

2021 · ARTICLE · en

Dealing with relational data always required significant computational resources, domain expertise and task-dependent feature engineering in order to incorporate structural information into predictive model. Nowadays, a family of automated graph feature engineering techniques have been proposed in different streams of literature. So-called graph embeddings provide a powerful tool to construct vectorized feature spaces for graphs and their components, such as nodes, edges and subgraphs under preserving inner graph properties. Using the constructed feature spaces, many machine learning problems on graphs can be solved via standard frameworks suitable for vectorized feature representation. Our survey aims to describe the core concepts of graph embeddings, and provide several taxonomies for their description. First, we start with methodological approach, and extract three types of graph embedding models based on matrix factorization, random-walks and deep learning approaches. Next, we describe how different types of networks impact the ability to of models to incorporate structural and attributed data into a unified embedding. Going further, we perform a thorough evaluation of graph embedding applications to machine learning problems on graphs, among which are node classification, link prediction, clustering, visualization, compression, and a family of the whole graph embedding algorithms suitable for graph classification, similarity and alignment problems. Finally, we overview the existing applications of graph embeddings to computer science domains, formulate open problems and provide experiment results, explaining how different embedding and graph properties are connected to the four classic machine learning problems on graphs, such as node classification, link prediction, clustering and graph visualization. As a result, our survey covers a new rapidly growing field of network feature engineering, presents an in-depth analysis of models based on network types, and overviews a wide range of applications to machine learning problems on graphs.

Fault detection in Tennessee Eastman process with temporal deep learning models

2021 · ARTICLE · en

Automated early process fault detection and prediction remains a challenging problem in industrial processes. Traditionally it has been done by multivariate statistical analysis of sensor readings and, more recently, with the help of machine learning methods. The quality of machine learning models strongly depends on feature engineering, that in turn heavily relies on expertise of the process engineers and model developers. With the recent advent of deep learning neural network methods and abundance of available sensor data, it became possible to develop advanced approaches to early fault detection and prediction that do not require feature engineering and provide more accurate and timely results. In this paper we investigate a wide range of recurrent and convolutional architectures on the publicly available simulated Tennessee Eastman Process extended TEP dataset for the fault detection in chemical processes. We have selected the best architecture for the task and proposed a novel temporal CNN1D2D architecture that achieves overall better performance on the dataset than any referenced method. We have also proposed to use Generative Adversarial Network GAN to extend and enrich data used in training.