OTA UPDATES ON ESP32 WITH EC25

OTA UPDATES ON ESP32 WITH EC25 #

Introduction #

The ESP32 microcontroller, in conjunction with the EC25 module, enables over-the-air (OTA) firmware updates. By storing the latest firmware on a GitHub repository, the ESP32 can periodically check for updates and download them via GPRS, ensuring that the device always runs the most recent version of the firmware. This process eliminates the need for physical access to the device and allows for efficient deployment of new features and security patches.

Project Overview #

The firmware update process works as follows:

• Current Version Check: The ESP32 checks the version.txt file in the GitHub repository to determine the latest firmware version.
• New Firmware Detection: If a new version is detected (i.e., different from the current firmware version running on the ESP32), the device downloads the latest (firmware_vX.X.X.bin) firmware file.
• OTA Update: The ESP32 initiates the OTA update and installs the new firmware. Once the update is complete, the ESP32 restarts and runs the new firmware.

Branch Setup

• The release branch of your GitHub repository should contain:
• The latest firmware file (firmware_vX.X.X.bin)
• A version.txt file that stores the version number of the latest firmware.

/release/
├── firmware_v1.0.1.bin
├── version.txt

• version.txt: This file should contain only the version number of the latest firmware. e.g., 1.0.1

Key components

• Quectel EC25 Module: Responsible for connecting to the internet via GPRS and making HTTPS requests.
• Update.h Library: Handles the Over-the-Air(OTA) process.
• Root CA Certificate: As GitHub uses HTTPS, the CA root certificate for raw.githubusercontent.com must be uploaded to the EC25 module to ensure secure communication. This is required since the OTA process downloads the firmware from a secure GitHub URL. The cert.h file in this repository contains the CA root certificate.

Software Setup #

A. Install the ESP32 board in Arduino IDE:

• a. Go to File > Preferences.
• b. Go to Tools > Boards > Boards Manager, search for “ESP32”, and install.
• c. OTA on ESP32 (https://github.com/IndustrialArduino/OTA-on-ESP/tree/main/OTA_on_ESP32_over_EC25)

B. Install Required Libraries (Arduino):

Go to Sketch > Include Library > Manage Libraries and install:

• Update.h: For communication and OTA functionality
• cert.h: Contains the CA certificate needed for HTTPS communication.

C. Configure the code:

• Open the code provided in your Arduino IDE.
• Replace the GPRS credentials (apn, gprsUser) with your SIM card provider details.
• Set the correct firmware version file paths. (ex: https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/version.txt)

Code Explanation #

This Arduino code demonstrates how to implement an Over-The-Air (OTA) firmware update on an ESP32 using a EC25 modem to connect to the internet. The program fetches the latest firmware version from a GitHub repository, compares it with the current version running on the device, and downloads the new firmware if necessary. Here’s a breakdown of the key components of the code.

• Part 1

#include <Arduino.h>
#include <WiFi.h>
#include <Update.h>
#include "cert.h"
String gsm_send_serial(String command, int delay);
#define SerialMon Serial
#define SerialAT Serial1
#define GSM_PIN ""
#define UART_BAUD 115200
#define MODEM_TX 32
#define MODEM_RX 33
#define GSM_RESET 21
// Your GPRS credentials
const char apn[] = "dialogbb";
const char gprsUser[] = "";

This Part defines essential libraries, constants, and variables for an OTA firmware update project on an ESP32 using the EC25 module. It includes libraries for OTA process, GPRS credentials, and firmware update settings.

• Part 2

String current_version = "1.0.0";
String new_version;
String version_url = "https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/version.txt";
String firmware_url;
//variabls to blink without delay:
const int led1 = 2;
const int led2 = 12;
unsigned long previousMillis = 0;  // will store last time LED was updated
const long interval = 1000;        // interval at which to blink (milliseconds)
int ledState = LOW;                // ledState used to set the LED

This code initializes variables for firmware versions and LED control. It defines the URL of the text file containing the latest version number and sets up the HTTPS transport for communication with the EC25 module.

• Part 3

void setup() {
  // Set console baud rate
  Serial.begin(115200);
  delay(10);
  SerialAT.begin(UART_BAUD, SERIAL_8N1, MODEM_RX, MODEM_TX);
  delay(2000);
  pinMode(GSM_RESET, OUTPUT);
  digitalWrite(GSM_RESET, HIGH);  // RS-485
  delay(2000);
  pinMode(led1, OUTPUT);
  pinMode(led2, OUTPUT);
  Init();
  connectToGPRS();
  connectTohttps();
}

This code initializes serial communication, resets the EC25 module, initialize the and establishes a GPRS connection. Then it connects to the GitHub through the https protocol. So, basically this is the setup of the program.

• Part 4

void loop() {
  if (checkForUpdate(firmware_url)) {
    performOTA(firmware_url);
  }
  delay(1000);
  //add the code need to run and this is an example program
  //loop to blink without delay
  unsigned long currentMillis = millis();
  if (currentMillis - previousMillis >= interval) {
    // save the last time you blinked the LED
    previousMillis = currentMillis;
    // if the LED is off turn it on and vice-versa:
    ledState = not(ledState);
    // set the LED with the ledState of the variable:
    digitalWrite(led1, ledState);
    digitalWrite(led2, ledState);
  }
}

This code loop() function continuously checks for firmware updates by calling the check ForUpdate() function. If an update is available, the performOTA() function is executed to download and install the new firmware. A delay of 1000 milliseconds is then added to prevent excessive polling. The code also includes an example program that blinks two LEDs without using delays, demonstrating the basic structure for adding other functionalities to the main loop.

• Part 5

void Init(void) {  // Connecting with the network and GPRS
  delay(5000);
  gsm_send_serial("AT+CFUN=1", 10000);
  gsm_send_serial("AT+CPIN?", 10000);
  gsm_send_serial("AT+CSQ", 1000);
  gsm_send_serial("AT+CREG?", 1000);
  gsm_send_serial("AT+COPS?", 1000);
  gsm_send_serial("AT+CGATT?", 1000);
  gsm_send_serial("AT+CPSI?", 500);
  String cmd = "AT+CGDCONT=1,\"IP\",\"" + String(apn) + "\"";
  gsm_send_serial(cmd, 1000);
  gsm_send_serial("AT+CGACT=1,1", 1000);
  gsm_send_serial("AT+CGATT?", 1000);
  gsm_send_serial("AT+CGPADDR=1", 500);
}
void connectToGPRS(void) {
  gsm_send_serial("AT+CGATT=1", 1000);
  String cmd = "AT+CGDCONT=1,\"IP\",\"" + String(apn) + "\"";
  gsm_send_serial(cmd, 1000);
  gsm_send_serial("AT+CGACT=1,1", 1000);
  gsm_send_serial("AT+CGPADDR=1", 500);
}
void connectTohttps(void) {
  int cert_length = root_ca.length();
  String ca_cert = "AT+QFUPL=\"RAM:github_ca.pem\"," + String(cert_length) + ",100";
  gsm_send_serial(ca_cert, 1000);
  delay(1000);
  gsm_send_serial(root_ca, 1000);
  delay(1000);
  gsm_send_serial("AT+QHTTPCFG=\"contextid\",1", 1000);
  gsm_send_serial("AT+QHTTPCFG=\"responseheader\",1", 1000);
  gsm_send_serial("AT+QHTTPCFG=\"sslctxid\",1", 1000);
  gsm_send_serial("AT+QSSLCFG=\"sslversion\",1,4", 1000);
  gsm_send_serial("AT+QSSLCFG=\"ciphersuite\",1,0xC02F", 1000);
  gsm_send_serial("AT+QSSLCFG=\"seclevel\",1,1", 1000);
  gsm_send_serial("AT+QSSLCFG=\"sni\",1,1", 1000);
  gsm_send_serial("AT+QSSLCFG=\"cacert\",1,\"RAM:github_ca.pem\"", 1000);
}

Network Initialization and GPRS Activation (Init and connectToGPRS)

• These functions initialize the EC25 modem, check SIM and network registration (AT+CPIN?, AT+CREG?), and activate GPRS by configuring and enabling the PDP context using:

• AT+CGDCONT=1,”IP”,”<APN>” – Sets the APN for internet access.
• AT+CGACT=1,1 – Activates the PDP context.
• AT+CGATT=1 – Attaches the device to the GPRS service.
• AT+CGPADDR=1 – Retrieves the assigned IP address

SSL Certificate Upload and HTTPS Configuration (connectTohttps)

• Uploads the root CA certificate (stored as a string root_ca) to the EC25’s RAM using:

• AT+QFUPL=”RAM:github_ca.pem”,<length>,100 followed by the actual certificate content.

• Configures HTTPS and SSL settings:

• Binds the HTTPS client to PDP context 1 and SSL context 1.
• Enables response headers, SNI (Server Name Indication), and sets TLS version (version 1.2).
• Specifies cipher suite and security level.
• Associates the uploaded CA cert to the SSL context using:
• AT+QSSLCFG=”cacert”,1,”RAM:github_ca.pem”.

Separation of Logic for Modular OTA Communication

• Code is cleanly separated into:

• Init() → handles initial modem and GPRS setup after boot.
• connectToGPRS() → Connect to GPRS.
• connectTohttps() → sets up HTTPS with SSL using a root certificate.

• Part 6

// Check the version of the latest firmware uploaded to GitHub
bool checkForUpdate(String &firmware_url) {
  Serial.println("Making GET request securely...");
  // Ensure PDP context is active (optional but recommended)
  gsm_send_serial("AT+QIACT?", 1000);
  delay(100);
  gsm_send_serial("AT+QIACT=1", 2000);  // Reactivate if needed
  delay(300);
  // Clear SerialAT buffer
  while (SerialAT.available()) SerialAT.read();
  // Step 1: Prepare the URL
  gsm_send_serial("AT+QHTTPURL=" + String(version_url.length()) + ",80", 1000);
  delay(200);
  gsm_send_serial(version_url, 2000);
  bool got200 = false;
  int contentLen = -1;
  // Flush again
  while (SerialAT.available()) SerialAT.read();
  // Step 2: Send HTTP GET
  Serial.println("Send ->: AT+QHTTPGET=80");
  SerialAT.println("AT+QHTTPGET=80");
  unsigned long startTime = millis();
  const unsigned long httpTimeout = 8000;  // Wait max 8s
  String qhttpgetLine = "";
  while (millis() - startTime < httpTimeout) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) continue;
      Serial.println("[Modem Line] " + line);
      if (line.startsWith("+QHTTPGET:")) {
        qhttpgetLine = line;
        break;
      }
      if (line.indexOf("+QIURC: \"pdpdeact\"") >= 0) {
        Serial.println("[OTA] PDP deactivated! Reconnecting...");
        gsm_send_serial("AT+QIACT=1", 3000);
        return false;
      }
    }
  }
  if (qhttpgetLine.length() == 0) {
    Serial.println("[OTA] HTTP GET response not received.");
    return false;
  }

  // Step 3: Parse +QHTTPGET: 0,<status>,<len>
  int idx1 = qhttpgetLine.indexOf(',');
  int idx2 = qhttpgetLine.indexOf(',', idx1 + 1);
  if (idx1 == -1 || idx2 == -1) {
    Serial.println("[OTA] Malformed +QHTTPGET response");
    return false;
  }
  int statusCode = qhttpgetLine.substring(idx1 + 1, idx2).toInt();
  contentLen = qhttpgetLine.substring(idx2 + 1).toInt();
  if (statusCode == 200 && contentLen > 0) {
    got200 = true;
    Serial.println("[OTA] HTTP 200 OK. Content length: " + String(contentLen));
  } else {
    Serial.println("[OTA] HTTP GET failed. Status: " + String(statusCode));
    return false;
  }
  delay(300);  // Give modem time to buffer
  // Step 4: Issue QHTTPREAD
  Serial.println("Send ->: AT+QHTTPREAD=" + String(contentLen));
  SerialAT.println("AT+QHTTPREAD=" + String(contentLen));
  // Step 5: Wait for CONNECT
  bool gotConnect = false;
  unsigned long waitStart = millis();
  while (millis() - waitStart < 3000) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      Serial.println("[Modem Line] " + line);
      if (line.startsWith("CONNECT")) {
        gotConnect = true;
        break;
      }
    }
  }
  if (!gotConnect) {
    Serial.println("[OTA] Failed to get CONNECT, aborting");
    return false;
  }
  // Step 6: Skip headers and extract the version
  String version = "";
  bool foundEmptyLine = false;
  unsigned long readTimeout = millis() + 5000;
  while (millis() < readTimeout) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) {
        foundEmptyLine = true;  // Found end of headers
        continue;
      }
      if (foundEmptyLine) {
        version = line;  // First non-empty line after headers
        break;
      }
      Serial.println("[Header] " + line);  // Optional logging
      readTimeout = millis() + 1000;       // Extend timeout while reading
    }
  }
  version.trim();
  Serial.println("Extracted version: " + version);
  if (version.length() == 0) {
    Serial.println("[OTA] Failed to extract version string.");
    return false;
  }
  // Set global new_version
  new_version = version;
  // Step 7: Compare and set firmware URL
  if (new_version != current_version) {
    Serial.println("New version available. Updating...");
    firmware_url = "https://raw.githubusercontent.com/IndustrialArduino/OTA-on-ESP/release/firmware_v" + new_version + ".bin";
    Serial.println("Firmware URL: " + firmware_url);
    return true;
  } else {
    Serial.println("Already on latest version.");
    return false;
  }
}

This code checks for updates by making a GET request to the GitHub repository and extracting the latest version number from the response. It compares the current version with the available version and updates the firmware URL if a new version is found. The code also prints relevant information to the serial monitor, including the status code, response body, and current and available versions.

• Part 7

void performOTA(String firmware_url) {
  gsm_send_serial("AT+QHTTPURL=" + String(firmware_url.length()) + ",80", 1000);
  delay(100);
  gsm_send_serial(firmware_url, 2000);
  gsm_send_serial("AT+QHTTPGET=80", 1000);
  Serial.println("[OTA] Waiting for +QHTTPGET response...");
  long contentLength = -1;
  unsigned long timeout = millis();
  while (millis() - timeout < 5000) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) continue;
      Serial.println("[Modem Line] " + line);
      if (line.startsWith("+QHTTPGET:")) {
        int firstComma = line.indexOf(',');
        int secondComma = line.indexOf(',', firstComma + 1);
        if (firstComma != -1 && secondComma != -1) {
          String lenStr = line.substring(secondComma + 1);
          contentLength = lenStr.toInt();
          Serial.print("[OTA] Content-Length: ");
          Serial.println(contentLength);
        }
      }
      if (line == "OK") break;
    }
    delay(10);
  }
  Serial.println("[OTA] HTTPS GET sent");
  // Save response to RAM file
  gsm_send_serial("AT+QHTTPREADFILE=\"RAM:firmware.bin\",80", 1000);
  // Wait for final confirmation and avoid overlap
  unsigned long readfileTimeout = millis();
  while (millis() - readfileTimeout < 5000) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) continue;
      Serial.println("[READFILE] " + line);
      if (line.startsWith("+QHTTPREADFILE:")) break;
    }
    delay(10);
  }

  // Clear SerialAT buffer
  while (SerialAT.available()) SerialAT.read();
  // Send QFLST directly
  SerialAT.println("AT+QFLST=\"RAM:firmware.bin\"");
  long ramFileSize = 0;
  timeout = millis();
  while (millis() - timeout < 5000) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) continue;
      Serial.println("[OTA Raw] " + line);
      // Find +QFLST line
      if (line.startsWith("+QFLST:")) {
        int commaIdx = line.lastIndexOf(',');
        if (commaIdx != -1) {
          String sizeStr = line.substring(commaIdx + 1);
          sizeStr.trim();
          ramFileSize = sizeStr.toInt();
          break;
        }
      }
    }
    delay(10);
  }
  Serial.println("[OTA] File size: " + String(ramFileSize));
  if (ramFileSize <= 0) {
    Serial.println("[OTA] ERROR: Invalid file size.");
    return;
  }
  int headerSize = ramFileSize - contentLength;
  if (headerSize <= 0 || headerSize > ramFileSize) {
    Serial.println("[OTA] Invalid header size!");
    return;
  }
  Serial.println("[OTA] Header size: " + String(headerSize));
  // Clear SerialAT buffer before command
  while (SerialAT.available()) SerialAT.read();
  // Send QFOPEN directly
  SerialAT.println("AT+QFOPEN=\"RAM:firmware.bin\",0");
  int fileHandle = -1;
  unsigned long handleTimeout = millis();
  while (millis() - handleTimeout < 5000) {
    if (SerialAT.available()) {
      String line = SerialAT.readStringUntil('\n');
      line.trim();
      if (line.length() == 0) continue;
      Serial.println("[OTA Raw] " + line);

      if (line.startsWith("+QFOPEN:")) {
        String handleStr = line.substring(line.indexOf(":") + 1);
        handleStr.trim();
        fileHandle = handleStr.toInt();
        break;
      }
    }
    delay(10);
  }
  Serial.println("[OTA] File handle: " + String(fileHandle));
  if (fileHandle <= 0) {
    Serial.println("[OTA] ERROR: Invalid file handle.");
    return;
  }
  // Seek to payload
  gsm_send_serial("AT+QFSEEK=" + String(fileHandle) + "," + String(headerSize) + ",0", 1000);
  delay(300);
  // Step 7: Begin OTA
  if (!Update.begin(contentLength)) {
    Serial.println("[OTA] Update.begin failed");
    return;
  }
  Serial.println("[OTA] Start writing...");


  size_t chunkSize = 1024;
  size_t totalWritten = 0;
  uint8_t buffer[1024];
  while (totalWritten < contentLength) {
    size_t bytesToRead = min(chunkSize, (size_t)(contentLength - totalWritten));
    SerialAT.println("AT+QFREAD=" + String(fileHandle) + "," + String(bytesToRead));
    // Wait for CONNECT (start of binary data)
    bool gotConnect = false;
    unsigned long startWait = millis();
    while (millis() - startWait < 2000) {
      if (SerialAT.available()) {
        String line = SerialAT.readStringUntil('\n');
        line.trim();
        if (line.startsWith("CONNECT")) {
          gotConnect = true;
          break;
        }
      }
      delay(1);
    }
    if (!gotConnect) {
      Serial.println("[OTA] Failed to get CONNECT");
      Update.abort();
      return;
    }

    // Read exactly bytesToRead bytes of binary data
    size_t readCount = 0;
    unsigned long lastReadTime = millis();
    while (readCount < bytesToRead && millis() - lastReadTime < 3000) {
      if (SerialAT.available()) {
        buffer[readCount++] = (uint8_t)SerialAT.read();
        lastReadTime = millis();
      } else {
        delay(1);
      }
    }
    if (readCount != bytesToRead) {
      Serial.println("[OTA] Incomplete read from modem");
      Update.abort();
      return;
    }

    // After reading data, wait for the final OK
    bool gotOK = false;
    startWait = millis();
    while (millis() - startWait < 2000) {
      if (SerialAT.available()) {
        String line = SerialAT.readStringUntil('\n');
        line.trim();
        if (line == "OK") {
          gotOK = true;
          break;
        }
      }
      delay(1);
    }
    if (!gotOK) {
      Serial.println("[OTA] Did not receive final OK after data");
      Update.abort();
      return;
    }

    // Write to flash
    size_t written = Update.write(buffer, readCount);
    if (written != readCount) {
      Serial.println("[OTA] Flash write mismatch");
      Update.abort();
      return;
    }

    totalWritten += written;
    Serial.printf("\r[OTA] Progress: %u / %u bytes", (unsigned)totalWritten, (unsigned)contentLength);
  }

  Serial.println("\n[OTA] Firmware write complete.");
  // Close the file
  SerialAT.println("AT+QFCLOSE=" + String(fileHandle));
  delay(500);
  // Finalize OTA update
  if (Update.end()) {
    Serial.println("[OTA] Update successful!");
    if (Update.isFinished()) {
      Serial.println("[OTA] Rebooting...");
      delay(300);
      ESP.restart();
    } else {
      Serial.println("[OTA] Update not finished!");
    }
  } else {
    Serial.println("[OTA] Update failed with error: " + String(Update.getError()));
  }
}

1. Download Firmware to RAM File

• Performs HTTPS GET of the .bin file and saves it to RAM:firmware.bin using AT+QHTTPREADFILE.
• Parses the response to get content length, file size, and calculates header size for skipping HTTP headers.

2. Flash Firmware to ESP32

• Opens the RAM file, skips headers using AT+QFSEEK, and reads firmware in 1024-byte chunks with AT+QFREAD.
• Streams each chunk to Update.write(). Once complete, closes file and finalizes OTA with Update.end() and restarts ESP if successful.

How OTA Update work #

1.Version Check: The ESP32 sends a request to the server (GitHub) to check the latest firmware version by fetching a simple text file version.txt.

• Current version on ESP32: 1.0.0.
• Latest version from the server: e.g., 1.1.0.

2.Download Firmware: If a newer version is detected, the firmware binary is downloaded securely via HTTPS. The firmware file is hosted at a URL like:https://github.com/IndustrialArduino/OTA-on-ESP/tree/release

3.Perform OTA Update: The downloaded binary is written to the ESP32’s memory. After a successful update, the ESP32 will restart automatically and run the new firmware.

Testing setup #

1.Upload the Initial Firmware

• Compile and upload the firmware to the ESP32 with the initial version (1.0.0).
• Verify that the device is connected to the GSM network.

2.Host a New Firmware

• After making changes to the code export the code as binary and then rename the binary file
• as the new version of the firmware (e.g., 1.1.0) and upload the binary file to your server.
• Update the version.txt file to reflect the new version number.
• Monitor the Update
• Open the Serial Monitor at 115200 baud.
  The ESP32 will:
• Connect to the GSM network.
• Fetch the latest version from the server.
• Compare the versions.
• If an update is available, it will download the firmware and install it.
• Reboot the device with the updated firmware.

3.Expected Output:

• ESP32 logs network connection and update status.
• If an update is available, the new firmware is downloaded and applied.

Making GET request securely...n[OTA] HTTP 200 OK. Content length: 112345n[OTA] Waiting for +QHTTPGET response...n[OTA] File size: 113123n[OTA] Header size: 778n[OTA] Start writing...n[OTA] Progress: 112345 / 112345 bytesn[OTA] Firmware write complete.n[OTA] Update successful!n[OTA] Rebooting...n

Error Handling #

Connection Failure: Ensure that the APN, username, and password are correct for your SIM card provider. Check for proper power supply to the Quetcel EC25 module

• HTTPS Failures: If there are SSL/HTTPS errors, verify that the root CA certificate is properly loaded in the cert.h file.
• OTA Failures: If the update fails, the device will continue running the current version. Ensure that the binary file is correct and available at the specified URL.

Troubleshooting #

ESP32 Doesn’t Connect to Cellular Network:

• Double-check the wiring between the Quetcel EC25 and the ESP32.
• Make sure the SIM card has an active data plan and is properly inserted.

OTA Update Fails:

• Check the size of the binary file and ensure there’s enough memory on the ESP32 for the update.
• Ensure the server is reachable, and the firmware URL is correct.

HTTPS Request Fails:

• Confirm that the root CA is correctly added for HTTPS communication.
• Check if the GitHub URL is accessible and the server isn’t blocking your requests.