Сакиркина Мария Александровна

Natural hazard database from Internet publications: text mining with a large language model

2026 · PREPRINT · en

Comprehensive data on natural hazards and their consequences are crucial for effective for risk assessment, adaptation planning, and emergency response. However, many countries face challenges with fragmented, inconsistent, and inaccessible data, particularly regarding local-scale events. To address this data gap in Russia, we developed an end-to-end processing pipeline that scrapes news from various online sources, including national newspapers, regional emergency office websites, and social media, and extracts hazard-related information into a structured database. The pipeline employs ChatGPT 4o mini for its cost effectiveness, quality performance and the ready-to-use mode. It identifies 14 information nuggets, including hazard type, location, dates, impact on infrastructure and people, and response measures. To enhance the result reliability, the outputs of a generative model are controlled by non-generative approaches of text processing. The resulting data are spatially located using OpenStreetMap service. In a test run, we scraped over 8.3 million primary news articles on 2018-2024, resulting in nearly 49,000 hazard-relevant texts that were related to 21,000 unique events across 18 hazard types. Evaluation against expert-labeled texts indicates recall and precision rates from 0.70 to 0.99 depending on the information rubric, with no instances of model hallucination. The combination of a ready-to-use large language model for text mining and the worldwide open-source geodata for geolocation supports the easily transfer of the pipeline to other regions and languages.