Considerations for a cloud-based system for IoT data acquisition from heterogeneous sensors

Abstract

Air pollution is a rising concern, demanding for low cost air quality monitoring systems. In this paper, we describe the back-end of an air quality monitoring system, developed in the context of the NanoSen-AQM [3, 4] project, a project with the goal of creating a real-time system that allows for a cost-effective, distributed and ubiquitous air quality monitoring. In particular, we describe the Data Aggregation Module. The NanoSen-AQM [3, 4] project is focused in the air quality monitoring using low-cost nanosensors, developed in the context of the project. The system will have at its core a cloud system that supports a mobile application, a web application and third party platforms. The cloud system starts from receiving the data in cloud system and adding its database, so the data can be monitored by the web application. The cloud system can be divided in two modules: 1) the Central System; and 2) the Data Aggregation Modules. The Data Aggregation Modules collect data from the sensors, acting as a buffer for the messages to be inserted in central system database. The Central System is responsible for storage, processing and data access. The web, mobile and third party applications fetch data from the aforementioned module. The Data Aggregation Modules receives data from the sensors, which then sends it to Central System to be stored. This module can be further divided in two sub-modules: 1) the Data Input; and 2) the Data Publishing Service. The Data Input Module receives data from the RESTFul[1] and MQTT[2] protocols. MQTT is a protocol developed for sensors and IoT thinking how the sensors are exposed to low quality connections. In cases where sensors have low resources and can’t handle the MQTT library, RESTFul is considered the best alternative. In both protocols, there is a Message Authentication Code (MAC) that validates each message integrity. The application that receives the message from the sensors, will also receive an hash. After the message is received in the application, the message is processed and accepted only if the hash is valid. The message then reaches the data publishing service that serves as a buffer to hold the messages before being inserted in the database. Meanwhile, in data publishing service, the messages need to be processed so they can be inserted in the database. The data publishing service uses Apache Kafka with Kafka Streams in order to serve as a buffer and data processing, respectively.