diff --git a/README.md b/README.md index d52129b..66d90d1 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,19 @@ -# Modbus to MQTT Bridge Service +# POE Sensor - Modbus to MQTT Bridge Service -A Python service that reads temperature and humidity data from a Modbus TCP server and publishes the data to an MQTT broker. +A robust Python service that reads temperature, humidity, and CO2 data from multiple Modbus TCP sensors and publishes the data to an MQTT broker with comprehensive monitoring and alerting capabilities. ## 📁 Project Structure ``` -POE/ -├── config.py # Configuration constants +POE-sensor/ +├── config.py # Configuration constants and sensor definitions ├── sensor_bridge.py # Core logic for Modbus and MQTT operations -├── main.py # Entry point +├── sensor_tracker.py # Sensor status tracking and alerting system +├── health_check.py # HTTP health check server (optional monitoring) +├── main.py # Entry point and application lifecycle management ├── requirements.txt # Python dependencies -├── README.md # This file -├── poe.py # Original single file (backup) -└── memory-bank/ # Memory bank directory +├── poe-sensor.nomad # Nomad deployment configuration +└── README.md # This documentation ``` ## 🚀 Quick Start @@ -25,13 +26,17 @@ pip install -r requirements.txt ### 2. Configure Settings Edit `config.py` to match your environment: ```python -# Modbus configuration -MODBUS_HOST = "10.84.48.153" -MODBUS_PORT = 505 +# Modbus sensor configuration +MODBUS_HOSTS = [ + {"ip": "10.84.60.31", "location": "Warehouse-B", "type": "temperature_humidity"}, + {"ip": "10.84.60.37", "location": "Office", "type": "cwt_co2"}, + # Add more sensors as needed +] # MQTT configuration MQTT_BROKER = "mqtt.service.mesh" -MQTT_TOPIC = "Temperature_Humidity" +MQTT_USERNAME = "relay" +MQTT_PASSWORD = "your_password" ``` ### 3. Run the Service @@ -41,130 +46,247 @@ python main.py ## 📊 Features -- **Modbus TCP Client**: Reads data from Modbus holding registers -- **MQTT Publisher**: Publishes sensor data to MQTT broker +### Core Functionality +- **Multi-Sensor Support**: Handles both temperature/humidity and CO2 sensors +- **Modbus TCP Client**: Reads data from multiple Modbus holding registers +- **MQTT Publisher**: Publishes sensor data to structured MQTT topics - **Data Processing**: Converts raw sensor values to calibrated readings - **Error Handling**: Robust error handling and retry mechanisms -- **Logging**: Comprehensive logging for monitoring and debugging -- **Health Check API**: HTTP endpoints for system health monitoring -- **Sensor Status Tracking**: Real-time tracking of individual sensor health -- **Alerting System**: Automatic alerts for sensor failures and recovery -- **Recovery Detection**: Detects when failed sensors come back online +- **Resource Management**: Automatic connection cleanup and garbage collection -## 📈 Data Format +### Monitoring & Alerting +- **Real-time Sensor Tracking**: Individual sensor health monitoring +- **Automatic Alerting**: MQTT alerts for sensor failures and recovery +- **Health Check API**: Optional HTTP endpoints for system monitoring +- **Comprehensive Logging**: Detailed logging for debugging and monitoring +- **Status Reporting**: Real-time sensor status via MQTT topics -The service publishes JSON data to the MQTT topic: +### Production Ready +- **Nomad Deployment**: Ready-to-deploy Nomad job configuration +- **Docker Support**: Containerized deployment with dependency management +- **Environment Configuration**: Environment variable support for flexible deployment +- **Graceful Shutdown**: Proper cleanup and signal handling + +## 📈 Data Format & Topics + +### MQTT Topic Structure +The service uses a hierarchical topic structure for better organization: + +``` +Location/{location_name}/{sensor_type}/Time +Location/{location_name}/{sensor_type}/Status +Location/{location_name}/{sensor_type}/Data/temperature +Location/{location_name}/{sensor_type}/Data/humidity +Location/{location_name}/{sensor_type}/Data/co2 # CO2 sensors only +Location/{location_name}/{sensor_type}/alerts # Alert messages +``` + +### Example Topics +``` +Location/Warehouse-B/temperature-humidity/Time → "2024-01-15 10:30:25" +Location/Warehouse-B/temperature-humidity/Status → "online" +Location/Warehouse-B/temperature-humidity/Data/temperature → "23.5" +Location/Warehouse-B/temperature-humidity/Data/humidity → "65.2" + +Location/Office/CO2-gas/Time → "2024-01-15 10:30:25" +Location/Office/CO2-gas/Status → "online" +Location/Office/CO2-gas/Data/temperature → "24.1" +Location/Office/CO2-gas/Data/humidity → "58.3" +Location/Office/CO2-gas/Data/co2 → "420" +``` + +### Alert Format ```json { - "time": "2024-01-15 10:30:25", - "location": "Office", - "temperature": 23.5, - "humidity": 65.2 + "alert_type": "sensor_failure", + "timestamp": "2024-01-15T10:30:25Z", + "sensor_id": "10.84.60.31_Warehouse-B", + "sensor_ip": "10.84.60.31", + "sensor_location": "Warehouse-B", + "sensor_type": "temperature_humidity", + "consecutive_failures": 3, + "last_error": "Connection timeout", + "severity": "critical" } ``` ## 🔧 Configuration -All configuration is centralized in `config.py`: +### Sensor Configuration +Add sensors to the `MODBUS_HOSTS` list in `config.py`: -| Parameter | Description | Default | -|-----------|-------------|---------| -| `MODBUS_HOST` | Modbus TCP server IP | "10.84.48.153" | -| `MODBUS_PORT` | Modbus TCP port | 505 | -| `MQTT_BROKER` | MQTT broker address | "mqtt.service.mesh" | -| `MQTT_TOPIC` | MQTT publish topic | "Temperature_Humidity" | -| `PUBLISH_INTERVAL` | Data publish interval (seconds) | 10 | -| `LOCATION` | Sensor location identifier | "Office" | - -## 📡 Sensor Mapping - -- **Register 0**: Temperature (raw value × 0.1 - 40) -- **Register 1**: Humidity (raw value × 0.1) - -## 🏥 Health Check & Monitoring - -### Health Check Endpoints - -The service provides HTTP endpoints for monitoring: - -- **http://localhost:8080/health** - Basic service health check -- **http://localhost:8080/sensors** - Detailed sensor status information - -### Alerting System - -The system automatically sends MQTT alerts for: - -- **Sensor Failures**: When a sensor fails 3 consecutive times -- **Sensor Recovery**: When a failed sensor comes back online - -Alert topics: -- `sensor-alerts` - Failure and recovery alerts -- `sensor-status/{location}/status` - Individual sensor status updates - -### Demo Monitoring Script - -Use the demo script to test monitoring features: - -```bash -# Check health endpoint -python demo_monitoring.py health - -# Check sensors status -python demo_monitoring.py sensors - -# Monitor system for 5 minutes -python demo_monitoring.py monitor 300 +```python +MODBUS_HOSTS = [ + # Temperature/Humidity sensors + {"ip": "10.84.60.31", "location": "Warehouse-B", "type": "temperature_humidity"}, + {"ip": "10.84.60.32", "location": "Warehouse-C", "type": "temperature_humidity"}, + + # CO2 sensors (CWT series) + {"ip": "10.84.60.37", "location": "Office", "type": "cwt_co2"}, +] ``` -### Configuration Options +### Environment Variables +The service supports environment variable configuration: -| Parameter | Description | Default | -|-----------|-------------|---------| -| `HEALTH_CHECK_ENABLED` | Enable/disable health check server | True | -| `HEALTH_CHECK_PORT` | HTTP server port | 8080 | -| `ALERTING_ENABLED` | Enable/disable MQTT alerts | True | +| Variable | Description | Default | +|----------|-------------|---------| +| `MQTT_BROKER` | MQTT broker address | "mqtt.service.mesh" | +| `MQTT_PORT` | MQTT broker port | 1883 | +| `MQTT_USERNAME` | MQTT username | "relay" | +| `MQTT_PASSWORD` | MQTT password | - | +| `PUBLISH_INTERVAL` | Data publish interval (seconds) | 10 | +| `HEALTH_CHECK_ENABLED` | Enable HTTP health server | true | +| `HEALTH_CHECK_PORT` | Health server port | 8080 | +| `ALERTING_ENABLED` | Enable MQTT alerts | true | | `SENSOR_TIMEOUT_THRESHOLD` | Failures before alert | 3 | | `RECOVERY_CONFIRMATION_COUNT` | Successes to confirm recovery | 2 | -## 🛠️ Maintenance +## 🏥 Health Check & Monitoring -### Running as a Service +### Optional Health Check Server +The service includes an optional HTTP health check server for monitoring: -For production deployment, consider running as a system service: +- **http://localhost:8080/health** - Basic service health status +- **http://localhost:8080/sensors** - Detailed sensor status information -```bash -# Example systemd service file -[Unit] -Description=Modbus MQTT Bridge -After=network.target - -[Service] -Type=simple -User=pi -WorkingDirectory=/path/to/POE -ExecStart=/usr/bin/python3 main.py -Restart=always - -[Install] -WantedBy=multi-user.target +### Health Response Example +```json +{ + "status": "healthy", + "timestamp": "2024-01-15T10:30:25Z", + "service": "poe-sensor", + "version": "1.0.0", + "sensors_total": 7, + "sensors_online": 6, + "sensors_health": 85.7 +} ``` -### Monitoring +### Sensor Status Response +```json +{ + "timestamp": "2024-01-15T10:30:25Z", + "total_sensors": 7, + "online_sensors": 6, + "offline_sensors": 1, + "sensors": { + "10.84.60.31_Warehouse-B": { + "ip": "10.84.60.31", + "location": "Warehouse-B", + "type": "temperature_humidity", + "status": "online", + "last_success": "2024-01-15T10:29:45Z", + "uptime_percentage": 98.5 + } + } +} +``` -The service provides comprehensive logging. Monitor the logs for: -- Connection status -- Data reading success/failure -- MQTT publish status -- Error conditions +## 🚀 Production Deployment + +### Nomad Deployment +The service is designed for production deployment using HashiCorp Nomad: + +```bash +# Deploy to Nomad cluster +nomad job run poe-sensor.nomad + +# Check deployment status +nomad job status poe-sensor + +# View logs +nomad alloc logs +``` + +### Key Deployment Features +- **No Health Check Dependencies**: Service is considered healthy when running +- **Fast Deployment**: Minimal startup time requirements +- **Auto-restart**: Automatic restart on application crashes +- **Resource Limits**: Configured CPU and memory limits +- **Log Management**: Automatic log rotation and retention + +### Docker Configuration +The Nomad job uses a pre-configured Docker image with: +- Python 3.x runtime +- Required system dependencies +- Automatic dependency installation +- Git-based source code deployment + +## 📡 Sensor Support + +### Temperature/Humidity Sensors +- **Register 0**: Temperature (formula: (125 - (-40)) * raw / 1650 - 40) +- **Register 1**: Humidity (formula: raw * 100 / 1000) +- **Output**: Temperature in °C, Humidity in %RH + +### CWT CO2 Sensors +- **Register 0**: Humidity (0.1%RH resolution) +- **Register 1**: Temperature (0.1°C resolution, 2's complement) +- **Register 2**: CO2 (1ppm resolution) +- **Output**: Temperature in °C, Humidity in %RH, CO2 in ppm + +## 🔧 Troubleshooting + +### Common Issues + +1. **Sensor Connection Failures** + - Check network connectivity to sensor IPs + - Verify Modbus port (default: 505) is accessible + - Check sensor power and network configuration + +2. **MQTT Connection Issues** + - Verify MQTT broker address and credentials + - Check network connectivity to MQTT broker + - Review MQTT broker logs for connection attempts + +3. **Health Check Not Responding** + - Health check server is optional and non-blocking + - Service will continue running even if health server fails + - Check port 8080 availability if health check is needed + +### Log Analysis +Monitor logs for key indicators: +- **Connection status**: Modbus and MQTT connection messages +- **Data reading**: Successful sensor data retrieval +- **Publishing status**: MQTT message publication confirmation +- **Error conditions**: Connection failures and retry attempts +- **Sensor health**: Failure/recovery notifications + +### Performance Monitoring +- **Cycle completion**: Monitor sensor processing cycles +- **Memory usage**: Automatic garbage collection logs +- **Alert frequency**: Sensor failure and recovery alerts +- **Uptime percentage**: Individual sensor reliability metrics ## 🤝 Contributing 1. Fork the repository -2. Create a feature branch -3. Make your changes -4. Test thoroughly -5. Submit a pull request +2. Create a feature branch (`git checkout -b feature/amazing-feature`) +3. Make your changes with proper testing +4. Commit your changes (`git commit -m 'Add amazing feature'`) +5. Push to the branch (`git push origin feature/amazing-feature`) +6. Open a Pull Request + +### Development Guidelines +- Follow Python PEP 8 style guidelines +- Add comprehensive logging for new features +- Test with multiple sensor types +- Update documentation for configuration changes +- Ensure backward compatibility when possible ## 📝 License -This project is licensed under the MIT License. +This project is licensed under the MIT License - see the LICENSE file for details. + +## 🆘 Support + +For issues and questions: +1. Check the troubleshooting section above +2. Review application logs for error details +3. Verify network connectivity and sensor configuration +4. Create an issue in the project repository with: + - Detailed error description + - Relevant log excerpts + - Environment configuration + - Steps to reproduce the issue \ No newline at end of file