ML-Based Anomaly Detection with Concept Drift
A streaming machine learning research project focused on detecting anomalies when behaviour changes over time.
Project Description
This project focuses on detecting anomalies in streaming or changing data environments where normal behaviour changes over time. This change is called concept drift. Traditional models trained on old data may fail when user behaviour, network traffic, sensor behaviour or transaction patterns evolve. River is especially relevant because it is designed for online machine learning on data streams, while scikit-learn provides baseline anomaly detection methods.
Problem
Many anomaly detection systems assume that the data distribution remains stable. In real-world systems such as cybersecurity, IoT, finance or user behaviour monitoring, patterns change over time. A model that was accurate yesterday may become unreliable after drift.
Solution
The project proposes an adaptive anomaly detection pipeline that continuously learns from streaming data, monitors drift, updates model behaviour and evaluates performance using precision, recall, F1-score, false alarm rate and detection delay.
Main Features
FeatureDescription
Streaming Data Input
Processes data continuously
Baseline Model
Isolation Forest, One-Class SVM or similar
Online Model
River-based adaptive model
Drift Detection
Detects behaviour change
Evaluation Metrics
Precision, recall, F1, false positives
Visualisation
Drift timeline and anomaly graphs
Use Cases
Cybersecurity, IoT, finance, system monitoring
Documentation Angle
This should be written as an MSc-level research proposal or experimental ML paper. Include literature review, dataset plan, methodology, baseline comparison, drift handling, evaluation framework and ethical/GDPR-safe dataset discussion.
