MQTT with SIMCOM SIM7500 Guide on NORVI GSM Series

GITHUB REPOSITORY : SIM7500_MQTT_NORVI_GSM #

NORVI devices are robust industrial controllers designed for IoT applications. When combined with the SIMCOM SIM7500 module, these devices can leverage cellular connectivity to communicate with cloud platforms securely using the MQTT protocol.

This guide provides an example implementation on how to use NORVI devices with the SIMCOM SIM7500 module to establish a connection with an MQTT broker using AT commands. The SIMCOM SIM7500 is a popular LTE module widely used in IoT applications for cellular communication. This guide will cover the necessary setup, code explanation, and essential AT commands.

MQTT PROTOCOL #

Understanding NORVI Devices with the SIM7500 Module #

NORVI devices are highly adaptable industrial IoT (Internet of Things) solutions designed to simplify the integration of sensors, actuators, and communication modules in various industrial applications. The integration of the SIM7500 module into NORVI devices enhances their capability by enabling cellular communication, which is crucial for remote monitoring, data collection, and control in areas without Wi-Fi or wired network access.

NORVI SIM7500 Devices

Setting Up NORVI GSM series SIM7500 to MQTT #

Here we’ll be considering the NORVI GSM series SIM7500 device as a Publisher and “MQTT.FX” software as the Subscriber. With the NORVI device, we’ll be storing the I/O values in the MQTT Subscriber and the data visualization platform.

Key features of NORVI devices with the SIM7500 module,

Industrial-grade design: Suitable for harsh environments.
Versatile I/O options: Digital and analog inputs/outputs for various sensors and actuators.
Cellular connectivity: LTE support via the SIM7500 module for remote monitoring and control.

Prerequisites

Before you begin, ensure you have the following,

NORVI Device with an integrated SIM7500 module
MQTT broker details (hostname, port, username, password)
MQTT.fx software for testing and monitoring
Arduino IDE and necessary libraries for programming the NORVI device.
Data visualization platform (e.g., DATACAKE)

Required Libraries

Arduino.h
Wire.h
WiFi.h
ArduinoJson.h

Sensitive Information Handling

Secret.h: This file should contain the MQTT username and password, which should be kept secure.

Hardware Setup #

Insert the registered SIM card to the module and upload the code.

Pin Connections

MODEM_TX (TX Pin)
MODEM_RX (RX Pin)
GSM_RESET (Reset Pin)
Digital Input Pins (D0, D1, D2, D3)

Understanding the Test program #

This code is for an ESP32-based NORVI device connected to a SIMCOM SIM7500 module via UART. It sets up a secure MQTT connection, subscribes to a topic to periodically publish the status of several digital inputs to an MQTT broker. The code also handles reconnection to both the network and the MQTT broker if connections are lost.
Download the example program.

1. Libraries and Includes

#include <Arduino.h> //Standard Arduino library.
#include <Wire.h>  //Library for I2C communication.
#include <WiFi.h>  // Include the WiFi library for MAC address
#include <ArduinoJson.h> //For handling JSON data.
#include "Secret.h" // Include the file to get the username and password of MQTT server

2. Function Declarations

String gsm_send_serial(String command, int delay);

3. Configuration and Definitions

#define TINY_GSM_MODEM_SIM7600
#define SerialMon Serial
#define SerialAT Serial1
#define GSM_PIN ""

// Your GPRS credentials, if any
const char apn[] = "dialogbb";
const char gprsUser[] = "";
const char gprsPass[] = "";

// MQTT details
String broker = "mqtt.sensoper.net";
String MQTTport="1881";

4. Pin Definitions

//Baud rate for serial communication with the SIM7500 module.
#define UART_BAUD 115200 

//Pin assignments for the ESP32.
#define MODEM_TX 32
#define MODEM_RX 33
#define GSM_RESET 21
#define D0 34
#define D1 35
#define D2 14
#define D3 13

5. MAC Address and Interval Definitions

#define MAC_ADDRESS_SIZE 18 // Assuming MAC address is in format "XX:XX:XX:XX:XX:XX"
byte mac[6]; 
String str_macAddress;

unsigned long prevMillis = 0;
const unsigned long interval = 60000; // Interval for sending messages

// Device-specific details
const char* deviceSerial = "***********";  // Replace with your device serial

6. MQTT Callback Function

This function checks if an incoming MQTT message is intended for the device by comparing MAC IDs. If they match, it processes the JSON payload and controls the relay (R0) based on the ‘state’ value.

void mqttCallback(char* topic, byte* payload, unsigned int len) {
  SerialMon.print("Message arrived [");
  SerialMon.print(topic);
  SerialMon.print("]: ");
  SerialMon.write(payload, len);
  SerialMon.println();

  // Extract serial number from the topic
  String topicStr = String(topic);
  int firstSlash = topicStr.indexOf('/');
  int lastSlash = topicStr.lastIndexOf('/');
  String MAC_ID = topicStr.substring(firstSlash + 1, lastSlash);

  SerialMon.print("MAC ID: ");
  SerialMon.println(MAC_ID);

  if (MAC_ID == deviceSerial) {
    // Decode the received message
    StaticJsonDocument<200> doc;
    DeserializationError error = deserializeJson(doc, payload, len);

    if (error) {
      SerialMon.print("deserializeJson() failed: ");
      SerialMon.println(error.c_str());
      return;
    }

    // Extract the payload
     bool state = doc["state"];

   if(state == 0){
      digitalWrite(R0,LOW);
   }else if(state == 1){
     digitalWrite(R0,HIGH);
   }
     
  } else {
    SerialMon.println("Received message for a different serial number");
  }
}

7. Setup Function

void setup(){
    // Set console baud rate
    Serial.begin(115200);
    delay(10);
    SerialAT.begin(UART_BAUD, SERIAL_8N1, MODEM_RX, MODEM_TX);

    delay(2000);
    pinMode(GSM_RESET, OUTPUT);
    digitalWrite(GSM_RESET,HIGH );   // RS-485 
    delay(2000);
    pinMode(D0, INPUT);
    pinMode(D1, INPUT);
    pinMode(D2, INPUT);
    pinMode(D3, INPUT);
    pinMode(R0, OUTPUT);

    Init();
    connectToGPRS();
    connectToMQTT();
  }

8. Main Loop
This loop ensures that the device regularly publishes its input states to the MQTT broker and stays connected to the network, handling any disconnections or errors as they occur.

void loop(){
   if (millis() - prevMillis >= interval) {
           prevMillis = millis();

    bool IN1 = digitalRead(D0);
    bool IN2 = digitalRead(D1);
    bool IN3 = digitalRead(D2);
    bool IN4 = digitalRead(D3);

    // Create JSON object
    StaticJsonDocument<200> doc;
    doc["D0"] = IN1 ? 1 : 0;
    doc["D1"] = IN2 ? 1 : 0;
    doc["D2"] = IN3 ? 1 : 0;
    doc["D3"] = IN4 ? 1 : 0;

    String jsonString;
    //char jsonBuffer[512];
    serializeJson(doc, jsonString);

    WiFi.macAddress(mac);
    
    str_macAddress = (String(mac[0] >> 4,  HEX) + String(mac[0] & 0x0F, HEX)) +
                     (String(mac[1] >> 4,  HEX) + String(mac[1] & 0x0F, HEX)) +
                     (String(mac[2] >> 4,  HEX) + String(mac[2] & 0x0F, HEX)) +
                     (String(mac[3] >> 4,  HEX) + String(mac[3] & 0x0F, HEX)) +
                     (String(mac[4] >> 4,  HEX) + String(mac[4] & 0x0F, HEX)) +
                     (String(mac[5] >> 4,  HEX) + String(mac[5] & 0x0F, HEX));
   str_macAddress.toUpperCase();
   String Digital_input = "NORVI/INPUTS/" +  str_macAddress;

   SerialMon.print("Published: ");
   SerialMon.println(jsonString);
   
   // Send a periodic "ping" message to verify MQTT connection
   String payload=(String)(millis()/1000L);
   String response = gsm_send_serial("AT+CMQTTTOPIC=0," +
    String(Digital_input.length()), 1000);
    response = gsm_send_serial(Digital_input, 1000);
    response = gsm_send_serial("AT+CMQTTPAYLOAD=0," +  
    String(jsonString.length()), 1000);
    response = gsm_send_serial(jsonString + "\x1A", 1000);
    response = gsm_send_serial("AT+CMQTTPUB=0,1,60", 1000);

    // Check if the publish command was successful
      if (response.indexOf("ERROR") != -1) {
            Serial.println("MQTT publish failed. Reconnecting...");
            connectToMQTT();
            if (!isGPRSConnected()) {
                Serial.println("GPRS connection lost. Reconnecting...");
                connectToGPRS();
                connectToMQTT();
            }
            if (!isNetworkConnected()) {
                Serial.println("Network connection lost. Reconnecting...");
                Init();
                connectToGPRS();
                connectToMQTT();
            }
        }
    }
    // Handle incoming MQTT messages
    handleIncomingMessages();
}

9. Handling Incoming Messages

void handleIncomingMessages() {
    while (SerialAT.available()) {
        String response = SerialAT.readStringUntil('\n');
        response.trim();

        if (response.startsWith("+CMQTTRXTOPIC")) {
            // Extract topic length
       int topicLength = response.substring(response.indexOf(",") + 1).toInt();
            SerialMon.print("Topic Length: ");
            SerialMon.println(topicLength);
            
            // Read the topic
            String topic = SerialAT.readStringUntil('\n');
            topic.trim();
            SerialMon.print("Topic: ");
            SerialMon.println(topic);

            // Confirm receipt of payload length
            response = SerialAT.readStringUntil('\n');
            response.trim();

            if (response.startsWith("+CMQTTRXPAYLOAD")) {
             int payloadLength = response.substring(response.indexOf(",") + 1).toInt();
                SerialMon.print("Payload Length: ");
                SerialMon.println(payloadLength);
                
                // Read the payload
                String payload = SerialAT.readStringUntil('\n');
                payload.trim();
                SerialMon.print("Payload: ");
                SerialMon.println(payload);

                // Ensure arrays are large enough to hold the topic and payload
                char topicArr[topicLength + 1];
                char payloadArr[payloadLength + 1];

                topic.toCharArray(topicArr, topicLength + 1);
                payload.toCharArray(payloadArr, payloadLength + 1);

                mqttCallback(topicArr, (byte*)payloadArr, payloadLength);
            }
        }
    }
}

10. Initializing the Network and GPRS

void Init(void){              // Connecting with the newtwork and GPRS
    delay(5000);
    gsm_send_serial("AT+CFUN=1", 10000);
    gsm_send_serial("AT+CPIN?", 10000);
    gsm_send_serial("AT+CSQ", 1000);
    gsm_send_serial("AT+CREG?", 1000);
    gsm_send_serial("AT+COPS?", 1000);
    gsm_send_serial("AT+CGATT?", 1000);
    gsm_send_serial("AT+CPSI?", 500);
    gsm_send_serial("AT+CGDCONT=1,\"IP\",\"dialogbb\"", 1000);
    gsm_send_serial("AT+CGACT=1,1", 1000);
    gsm_send_serial("AT+CGATT?", 1000);
    gsm_send_serial("AT+CGPADDR=1", 500);
    gsm_send_serial("AT+NETOPEN", 500);
    gsm_send_serial("AT+NETSTATE", 500);
}

11. Connecting to GPRS and MQTT

void connectToGPRS(void) {
    gsm_send_serial("AT+CGATT=1", 1000);
    gsm_send_serial("AT+CGDCONT=1,\"IP\",\"dialogbb\"", 1000);
    gsm_send_serial("AT+CGACT=1,1", 1000);
    gsm_send_serial("AT+CGPADDR=1", 500);
    gsm_send_serial("AT+NETOPEN", 500);
    gsm_send_serial("AT+NETSTATE", 500);
}

void connectToMQTT(void){
    gsm_send_serial("AT+CMQTTSTART", 1000);
    gsm_send_serial("AT+CMQTTACCQ=0,\"client test0\",0", 1000);
    gsm_send_serial("AT+CMQTTWILLTOPIC=0,2", 1000);
    gsm_send_serial("01\x1A", 1000);
    gsm_send_serial("AT+CMQTTWILLMSG=0,6,1", 1000);
    gsm_send_serial("qwerty\x1A", 1000);
  String command = "AT+CMQTTCONNECT=0,\"tcp://mqtt.sensoper.net:1881\",60,1,\"" + username + "\",\"" + password + "\"";
    gsm_send_serial(command, 10000);
    delay(2000);
    String downlinkTopic = "NORVI/+/OUTPUT";
    gsm_send_serial("AT+CMQTTSUB=0,14,1", 1000);
    gsm_send_serial(downlinkTopic + "\x1A", 1000);
    delay(2000);
}

12. Utility Functions

// Check if the network is connected by sending AT command
bool isNetworkConnected() {
    String response = gsm_send_serial("AT+CREG?", 3000);
    return (response.indexOf("+CREG: 0,1") != -1 || response.indexOf("+CREG: 0,5") != -1);
}

// Check if the GPRS is connected by sending AT command
bool isGPRSConnected() {
    String response = gsm_send_serial("AT+CGATT?", 3000);
    return (response.indexOf("+CGATT: 1") != -1);
}

// Send AT command and return the response
String gsm_send_serial(String command, int delay) {
    String buff_resp = "";
    Serial.println("Send ->: " + command);
    SerialAT.println(command);
    long wtimer = millis();
    while (wtimer + delay > millis()) {
        while (SerialAT.available()) {
            buff_resp += SerialAT.readString();
        }
    }
    Serial.println(buff_resp);
    return buff_resp;
}

Software Configurations #

Essential AT Commands #

After uploading the code to the NORVI device, it’s important to verify that the SIM card is correctly registered and the device is able to communicate with the network.
This can be done by checking the serial monitor for specific AT command responses.

Signal Quality Check (AT+CSQ),

Checks the signal quality of the cellular network.
Good Signal: An RSSI value of 18 indicates strong signal strength.
Weak Signal: A lower value, like 5, suggests weak signal strength, which could cause connection issues.

Network Registration Check (AT+CREG?),

Registered: The 1 indicates the device is registered on its home network.
Not Registered: A response of 0,0 means the device isn’t registered and isn’t searching for a network, possibly due to SIM or network issues.

Operator Selection Check (AT+COPS?),

This response shows that the device is connected to the operator “Dialog.”
The value “7” indicates the type of network (e.g., LTE).
If the operator details are missing or incorrect, it may indicate a problem with network registration or SIM card compatibility.

Query MQTT Connection Status(AT+QMTCONN?)

Queries the status of the MQTT connection.
The 3 suggests that the MQTT connection is established and connected.

Query Attach Status(AT+CGATT?)

This command output shows that the device is currently attached to the GPRS network.
If the response were +CGATT: 0, it would indicate that the device isn’t connected, which would prevent any data transfer over the cellular network.

Query Packet-Switched Service Information(AT+CPSI?)

This command is typically used to get information about the serving cell, including the network status, cell ID, and signal quality.
The ‘ERROR’ response suggests that the command might not be supported or a configuration issue. Some modules may require different configurations or initialization before using this command.

Define PDP Context(AT+CGDCONT=1,”IP”,”dialogbb”)

This command defines a PDP (Packet Data Protocol) context.
It specifies the context identifier (1 in this case), PDP type (IP), and the APN (Access Point Name), which is dialog bb.

Activate/Deactivate PDP Context(AT+CGACT=1,1)

This command activates the specified PDP context.
The first 1 indicates the context identifier, and the second 1 specifies that it should be activated.

Query PDP Address(AT+CGPADDR=1)

This command queries the IP address assigned to the specified PDP context.
‘1’ is the context identifier used when establishing the data connection.
“100.93.86.213” is the IP address assigned to the device.

Configure MQTT Receive Mode(AT+QMTCFG=”recv/mode”,0,0,1)

The parameters typically include the mode of receiving messages (QoS 0 or 1).
Indicates that there was an issue.
This could be due to an incorrect parameter or an unsupported feature

Open MQTT Connection(AT+QMTOPEN=0,*”,1881)

Opens an MQTT connection to the specified broker address (**) on port 1881.
0 Refers to the MQTT client index.
2 suggests that the connection is in progress.

Connect to MQTT Broker( AT+QMTCONN=0,*****,******,****** )

Connects to the MQTT broker with client ID, username , and password .

Subscribe to MQTT Topic( AT+QMTSUB=0,1,”NORVI/+/OUTPUT”,0)

Subscribes to the MQTT topic “NORVI/+/OUTPUT” with QoS level 0.
This response indicates that the subscription was successful.

Setting Up the MQTT Broker #

MQTT Broker Configuration
Use the MQTT.fx software as a Subscriber to set up the MQTT broker for the NORVI device. Below is the essential information required to configure the MQTT broker.

MQTT Broker URL (IP or domain)
Port number
Client ID (optional but recommended)
Username and Password (if authentication is required)
Topic(s) to publish/subscribe to

String Digital_input = "NORVI/INPUTS/" + str_macAddress; //MQTT Topic

Steps to Configure in MQTT.fx
Check this link for detailed instructions on how to configure the MQTT broker and the Subscriber.

Download MQTT.fx client and install it.
Open the MQTT.fx and click the Settings icon.
Click + to create a profile.
Enter the Connection Profile and General information.

Enter the User Credentials information

After completing the above steps, click Apply > OK to save.
Then, select the name of the file just created in the profile box and click Connect.
If the round icon in the top-right corner is green, the connection to IoT Hub is successful, and publishing and subscribing operations can be performed.

Select the Subscribe tab in the client, scan and select the topic name, and click Subscribe to subscribe to the topic.

The subscribing result can be seen in the bottom right corner.

Integration with Data Visualization Platform #

The NORVI Devices can be connected to an IoT platform, allowing users to visualize data collected by the NORVI devices. The platform used in this case is DATACAKE. Here’s how to integrate the NORVI devices with DATACAKE.

Access the DATACAKE from this link and navigate the Datacake dashboard. Select the “Add Devices”.

For the MQTT integration select the API. This will add the device to the Datacake account.

Click on Configuration Scroll down a bit and go to the new panel “MQTT Configuration”. Press on “Add New MQTT Broker”. Fill in the server details and add.

Then link the MQTT device on the broker to the device in the DATACAKE by providing the MQTT upline decoder. Subscribe to the topic on the MQTT broker and write a payload decoder.

This decoder function is designed to decode incoming messages from a device and format them into a structure that can be sent to the Datacake API for display or storage.

This function starts by converting the incoming payload, Extracting the MAC ID from the Topic, presumably in JSON format, into a JavaScript object. It uses JSON.parse() to accomplish this.

function Decoder(topic, payload) {
   var mac_id = topic.split("/")[2];
   payload = JSON.parse(payload);

Convert the string representations of the digital input states (which might be “true” or “false”) into integers (0 or 1).

 // Convert string "true"/"false" to integers (0 or 1)
    var D0 = JSON.parse(payload.D0) ;
    var D1 = JSON.parse(payload.D1) ;
    var D2 = JSON.parse(payload.D2);
    var D3 = JSON.parse(payload.D3);

Return an array of objects where each object contains:
device: A serial number or device ID (hard coded as “N123O456R789VI”).
field: The name of the digital input pin (“D0”, “D1”, “D2”, “D3”).
value: The state of the corresponding digital input pin (0 or 1).

//if(mac_id =="************") 
 return [
        {   device: "Serial Number", // Serial Number or Device ID
            field: "D0",
            value: D0
        },
        {   device: "Serial Number", // Serial Number or Device ID
            field: "D1",
            value: D1
        },
        {   device: "Serial Number", // Serial Number or Device ID
            field: "D2",
            value: D2
        },
        {   device: "Serial Number", // Serial Number or Device ID
            field: "D3",
            value: D3
        }
    ];
}

“Try Decoder Function” allows the decoder function to be tested to verify its functionality before deploying it. By entering sample data and observing the output, you can verify that the decoder is correctly parsing incoming payloads and formatting the data in a way that can be sent to the Datacake API.

Decoder function to verify functionality

To create a first database field, click on the “Add Field”. This will open another modal asking for some required input for the fields.
When typing the name for that field, the identifier auto-fills. This identifier is unique and cannot be changed. it’s required for accessing the field from within the payload decoder or API.