System Overview
FLOPY-NET is designed as a modular, policy-driven platform for studying federated learning in realistic network environments. This document provides a comprehensive overview of the system architecture, component interactions, and design principles.
High-Level Architecture
Design Principles
1. Policy-Driven Architecture
The Policy Engine is the central nervous system of FLOPY-NET, ensuring that all components operate according to defined security, performance, and governance rules.
Key Features:
- Centralized policy definition and enforcement (Flask REST API on port 5000)
- Real-time compliance monitoring with event buffer system
- Dynamic policy updates without system restart
- SQLite and in-memory storage backends
- Custom policy functions in
config/policy_functions/
2. Modular Component Design
Each component is designed as an independent service with well-defined interfaces:
3. Observable and Controllable Systems
Every component exposes metrics, logs, and control interfaces:
- Metrics: Performance, health, and business metrics
- Events: System events and state changes
- Controls: REST APIs for configuration and control
- Health Checks: Liveness and readiness probes
4. Research-First Design
The platform is optimized for research workflows:
- Reproducible Experiments: Deterministic seeding and configuration
- Extensible Frameworks: Plugin architecture for custom algorithms
- Data Export: Comprehensive data export for analysis
- Scenario Management: Pre-defined and custom scenarios
Component Interactions
Startup Sequence
Experiment Lifecycle
Data Flow Architecture
Metrics Collection Flow
Policy Enforcement Flow
Security Architecture
Multi-Layer Security Model
FLOPY-NET implements security at multiple layers:
- Network Layer: SDN-based network isolation and monitoring
- Application Layer: Policy-based access control and validation
- Data Layer: Encryption at rest and in transit
- Component Layer: Service-to-service authentication
Scalability Architecture
Horizontal Scaling
Performance Characteristics
Throughput Metrics
| Component | Metric | Target | Maximum |
|---|---|---|---|
| Dashboard API | Requests/sec | 1,000 | 10,000 |
| Collector | Metrics/sec | 10,000 | 100,000 |
| Policy Engine | Evaluations/sec | 5,000 | 50,000 |
| FL Framework | Clients | 100 | 1,000 |
Latency Targets
| Operation | P50 | P95 | P99 |
|---|---|---|---|
| API Response | 50ms | 200ms | 500ms |
| Metric Collection | 10ms | 50ms | 100ms |
| Policy Evaluation | 5ms | 20ms | 50ms |
| FL Round Start | 1s | 5s | 10s |
Deployment Architecture
Development Environment
Production Environment
Technology Stack
Core Technologies
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | React 18 + TypeScript | Interactive dashboard |
| Backend | FastAPI + Python | REST API services |
| Database | PostgreSQL + InfluxDB | Metrics and metadata storage |
| Message Queue | Redis | Event streaming |
| Container | Docker + Compose | Service orchestration |
| Network | GNS3 + OpenFlow | Network simulation |
ML/FL Technologies
| Component | Technology | Purpose |
|---|---|---|
| FL Framework | Custom + PyTorch | Federated learning implementation |
| Model Training | PyTorch | Deep learning models |
| Data Handling | Pandas + NumPy | Data preprocessing |
| Visualization | Plotly + D3.js | Interactive charts |
Configuration Management
Configuration Hierarchy
Configuration Files Structure
config/
├── system/
│ ├── collector_config.json
│ ├── policy_config.json
│ └── server_config.json
├── scenarios/
│ ├── basic_fl.json
│ ├── non_iid.json
│ └── byzantine.json
├── policies/
│ ├── security_policies.json
│ ├── performance_policies.json
│ └── compliance_policies.json
└── network/
├── topology_configs/
└── gns3_templates/
Monitoring and Observability
Metrics Collection
FLOPY-NET collects four types of metrics:
- Business Metrics: FL accuracy, convergence, client participation
- Performance Metrics: Latency, throughput, resource utilization
- Infrastructure Metrics: CPU, memory, disk, network usage
- Security Metrics: Policy violations, authentication failures
Logging Strategy
This architecture ensures that FLOPY-NET provides a robust, scalable, and research-friendly platform for studying federated learning in realistic network environments.