This paper presents an adaptive ensemble framework for real-time anomaly detection in large-scale data streams, addressing the challenges of concept drift, high-velocity data processing, and computational efficiency in modern distributed systems. We propose a Hybrid Statistical-Machine Learning Anomaly Detection (HSML-AD) algorithm that combines sliding window-based statistical analysis with incremental machine learning techniques. The framework employs a three-tier architecture: (1) lightweight statistical pre-filtering using modified Z-score and interquartile range methods, (2) adaptive feature extraction through exponential moving averages, and (3) ensemble classification using online random forest with dynamic weight adjustment based on recent prediction accuracy. Experimental evaluation on five benchmark datasets (KDD Cup 99, NSL-KDD, CICIDS2017, Yahoo S5, and Numenta Anomaly Benchmark) demonstrates that HSML-AD achieves an average F1-score of 94.3%, precision of 93.8%, and recall of 94.7%, outperforming baseline methods including Isolation Forest (F1: 87.2%), LSTM-Autoencoder (F1: 89.6%), and SPOT (F1: 86.4%). The algorithm maintains processing throughput of 127,000 records per second with average latency of 7.8 milliseconds on commodity hardware. The novelty lies in the adaptive weight mechanism that dynamically adjusts ensemble components based on data stream characteristics and recent performance, coupled with a memory-efficient incremental learning strategy that limits model size to 45 MB while maintaining detection accuracy. The proposed framework is applicable to network intrusion detection, IoT sensor monitoring, financial fraud detection, and industrial system health monitoring, particularly in resource-constrained environments requiring real-time processing.
| Mualliflar | Karshiyev, Zaynidin, Sattarov, Mirzabek, Erkinov, Farkhodjon |
|---|---|
| Jurnal | Techscience.uz - техника фанлари долзарб масалалри |
| Nashr sanasi | 2025-12-27 |
| Jild | 3 |
| Son | 12 |
| Betlar | 74-93 |
| Til | Ingliz |
| DOI | 10.47390/ts-v3i12y2025n09 |
DOI: 10.47390/ts-v3i12y2025n09 · Maqolaning asl sahifasi
anomaly detection, data streams, ensemble learning, concept drift, real-time processing, adaptive algorithms, machine learning., anomaliyalarni aniqlash, ma’lumotlar oqimi, ansambl o‘qitish, kontseptual siljish, real vaqt rejimida qayta ishlash, moslashuvchan algoritmlar, mashinali o‘qitish.
This article is devoted to the study of the corrosion process of metals in various aggressive environments and the identification of effective protection strategies. The problem is associated with the reduction of the…
The article developed and compared MIMO models of classical, azeotropic, and extractive rectification. Key differences in dynamics, thermodynamics, and controllability are shown. The need to apply predictive control for…
The article examines an approach based on the construction of intelligent virtual analyzers, which calculate the values of difficult-to-measure quality indicators based on the current readings of technological parameter…
This article uses clustering analysis, an important area of data mining in the field of education (Educational Data Mining - EDM). The article proposes a methodology for analyzing data on student academic performance…
This article examines the key factors determining the quality of deodorized vegetable oils: the physicochemical characteristics of the raw materials, process parameters, equipment design features, and the level of…
This paper addresses the development of an integrated intelligent control system based on artificial intelligence for optimizing temperature, pressure, and combustion processes in gas-fired industrial furnaces. The main…
This paper considers modern methods of colorimetric control of liquid food products. Particular attention is paid to spectrophotometric approaches that provide high accuracy, reproducibility, and the possibility of…
Artificial Intelligence (AI) is an interdisciplinary field focused on developing methods and technologies that replicate human intellectual behavior. Choosing the appropriate programming language is crucial for…
Wireless Sensor Networks (WSNs) play an increasingly central role in environmental monitoring, industrial automation, agricultural sensing, and many emerging IoT systems. Due to strict energy limitations of sensor…
This article discusses methods for intelligent modeling of industrial technological systems and the optimization of real-time control strategies. Within the framework of the Industry 4.0 concept, the theoretical and…
Techscience.uz - техника фанлари долзарб масалалри — barcha maqolalar