Collecting Temperature and Humidity with an Arduino Rev2 Wifi Board, and sending them over wifi to a server. The problem we are facing is that the loop fails after 300 loops, whether the interval time is 5s or 60s(code works for 22,000 seconds) or 100s(code works for 38,000 seconds then). At the moment I have set it up to 100s to increase the ov…

gistfile1.txt

//**************************************************************************************************************************
//**************************************************************************************************************************
// CLEANER-STABLER-DATA-COLLECTION
//**************************************************************************************************************************
//**************************************************************************************************************************


// Wifi libraries
#include <SPI.h>
#include <WiFiNINA.h>

// DHT22 sensor library
#include <DHT.h>


////////////////////////////////////////////////////
// WIFI SPECIFIC VARIABLES
////////////////////////////////////////////////////

char ssid[] = "XXX";              //  your network SSID (name)
char pass[] = "YYY";            // your network password

char server[] = "www.hostname.com";

// WIFI status can be either : WL_IDLE_STATUS or WL_CONNECTED when connected
int status = WL_IDLE_STATUS;

// Initialize the Wifi client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
WiFiClient client;

char dataFileName[12]; // data collection filename


////////////////////////////////////////////////////
// TRYING TO COUNTER THE FACT THE LOOP STOPS AFTER 380 LOOPS
////////////////////////////////////////////////////

// **** Initialization ****
// This block is towards the top of the file, in the initialization.

// Declare a string format for the whole request content, as a constant.
const char requestFormat[] = "GET /arduino/data.php?fi=%s&ti=%ld&te=%d&hu=%d&te2=%d&hu2=%d HTTP/1.1\n\
Host: www.hostname.com\n\
Connection: close\n";

// And declare a buffer meant to receive the actual request string.
// 200 is the estimated size of the formatted request, with real values replacing place holders.
// If the number is too low it will cause problems!

char requestBuffer[200];

// **** End initialization **


////////////////////////////////////////////////////
// SENSOR SPECIFIC VARIABLES
////////////////////////////////////////////////////

// TOP SENSOR IS CONNECTED TO 7
//Constants
#define DHTPIN 7     // what pin we're connected to
#define DHTTYPE DHT22   // DHT 22  (AM2302)
DHT dht(DHTPIN, DHTTYPE); //// Initialize DHT sensor for normal 16mhz Arduino

//Variables for sensor 1
int hu;  //Stores humidity value
int te; //Stores temperature value

// BOTTOM SENSOR IS CONNECTED TO 6
//Constants
#define DHTPIN2 6     // what pin we're connected to
#define DHTTYPE2 DHT22   // DHT 22  (AM2302)
DHT dht2(DHTPIN2, DHTTYPE2); //// Initialize DHT sensor for normal 16mhz Arduino

//Variables for sensor 2
int hu2;  //Stores humidity value
int te2; //Stores temperature value

// warning MILLIS() can't use INT variabbles, or it will spit out negative values sometimes
unsigned long lastCollectionTime = 0;
unsigned long intervalTime = 5; // time between two data collection

//**************************************************************************************************************************
// SETUP
//**************************************************************************************************************************


void setup() {
  
  Serial.begin(9600);
  //wait for serial to be ready
  while (!Serial); 

  Serial.println("*********************************************");
  Serial.println("*******THE-PASTA-DRYING-DATA-LOGGER**********");
  Serial.println("*********************************************");


  ///////////////////////////////////////////////////////////////
  // SETUP : SENSORS
  ///////////////////////////////////////////////////////////////
  
  // DHT22 Sensor 1 setup
  dht.begin();

  // DHT22 Sensor 2 setup
  dht2.begin();

  ///////////////////////////////////////////////////////////////
  // SETUP : LED 
  ///////////////////////////////////////////////////////////////

  // 13 is the blue led used for WIFI ON OFF
  pinMode(13, OUTPUT);
  
  // 12 is the white led used for DATA COM
  pinMode(12, OUTPUT);

  // 11 is the green LED use for TIME COM
  pinMode(11, OUTPUT);

  // SAY HELLO
  digitalWrite(13, HIGH);//blue
  digitalWrite(12, HIGH);//white
  digitalWrite(11, HIGH);//green
  delay(1000);
  digitalWrite(13, LOW);//blue
  digitalWrite(12, LOW);//white
  digitalWrite(11, LOW);//green

  ///////////////////////////////////////////////////////////////
  // SETUP : WIFI
  ///////////////////////////////////////////////////////////////
  
  while (status != WL_CONNECTED) {
    
    printTimeElapsed();
    
    Serial.print("Trying to connect to : ");
    Serial.println(ssid);
    status = WiFi.begin(ssid, pass);
    delay(5000); //wait 5s for connection
  }

  //printTimeElapsed();

  Serial.println("Connected to Wifi");
  digitalWrite(13, HIGH);//blue

  ///////////////////////////////////////////////////////////////
  // SETUP : Display WIFI INFORMATION
  ///////////////////////////////////////////////////////////////
  
  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    Serial.println("Firmware : Please upgrade");
  }else{
    Serial.println("Firmware : Up to date");
  }
  
  printCurrentNet();
  printWiFiData();

  //printTimeElapsed();


  ///////////////////////////////////////////////////////////////
  // SETUP : Create data filename using TIME
  ///////////////////////////////////////////////////////////////

  Serial.println("Generating Random Data Filename...");

  long randNumber;
  randomSeed(analogRead(0));
  randNumber = random(100000,1000000);
  
  Serial.print("Random number : ");
  Serial.println(randNumber);

  sprintf(dataFileName,"data%lu",randNumber);

  Serial.print("dataFileName= ");
  Serial.println(dataFileName);

  //printTimeElapsed();
    
}



//**************************************************************************************************************************
// LOOP
//**************************************************************************************************************************



void loop() {

  ///////////////////////////////////
  // LOOP :COLLECT DATA USING SENSOR
  ///////////////////////////////////
  //delay(2000); // at least 2000ms between two readings of sensor

  unsigned long t1;
  t1 = millis()/1000; // this is the time at which data will be collected, so t1 needs to be sent va GET

  unsigned long t2;
  t2 = lastCollectionTime + intervalTime; //collect data every XXs 

  if(t1 >= t2){ //is it time to collect data ?

    // Update the lastCollectionTime
    lastCollectionTime = t1;

    Serial.println("-");
    Serial.println("TIME TO COLLECT DATA !");
    //printTimeElapsed();
    blinkLed(12);

    //Read data and multiply by 100 and store it to variables hu and te
    hu = 100*dht.readHumidity();
    te = 100*dht.readTemperature();
  
    //Print temp and humidity values to serial monitor
    Serial.print("DHT22 Sensor #1: Reading Temperature: ");
    Serial.print(te);
    Serial.print("°C, Humidity: ");
    Serial.print(hu);
    Serial.println("%");

    //Read data and multiply by 100 and store it to variables hu and te
    hu2 = 100*dht2.readHumidity();
    te2 = 100*dht2.readTemperature();
  
    //Print temp and humidity values to serial monitor
    Serial.print("DHT22 Sensor #2: Reading Temperature: ");
    Serial.print(te2);
    Serial.print("°C, Humidity: ");
    Serial.print(hu2);
    Serial.println("%");

    // time to TRY to send data to the server

    Serial.println("Trying to send this data to the server...");
    
    // wifi check first
    Serial.print(">> CHECKING WiFi.status() = ");
    Serial.println(WiFi.status());
    Serial.print(">> WiFi.status() MEANING : ");

    switch (WiFi.status()) {
      case 3:
        Serial.println("Connected");
        break;
      case 4:
        Serial.println("Connection Failed");
        status = WL_IDLE_STATUS;
        WiFi.disconnect();
        digitalWrite(13, LOW);//blue
        break;
      case 5:
        Serial.println("Connection Lost");
        status = WL_IDLE_STATUS;
        WiFi.disconnect();
        digitalWrite(13, LOW);//blue
        break;
      case 6:
        Serial.println("Disconnected");
        status = WL_IDLE_STATUS;
        WiFi.disconnect();
        digitalWrite(13, LOW);//blue
        break;
      case 255:
        Serial.println("No Shield");
        status = WL_IDLE_STATUS;
        WiFi.disconnect();
        digitalWrite(13, LOW);//blue
        break;
      default:
        Serial.println("???");
        status = WL_IDLE_STATUS;
        WiFi.disconnect();
        digitalWrite(13, LOW);//blue
        break;
      }

    while (status != WL_CONNECTED) {
      // TRYING TO RECONNECT
      Serial.println("(!) WiFi Connection is failing !");
      Serial.print("(!) Trying to reconnect to : ");
      Serial.println(ssid);
      status = WiFi.begin(ssid, pass);
      delay(5000); //wait 5s for connection
      }

    Serial.println("***Connected to Wifi***");
    digitalWrite(13, HIGH);//blue

    Serial.println("Connecting to server...");
      
    // if you get a connection, report back via serial:
    if (client.connectSSL(server, 443)) {
    
      Serial.println("Success ! SSL-Connected to server");
      digitalWrite(11, HIGH);//green

      // This uses requestFormat as format,
      // replaces all placeholders by the values passed as arguments (dataFileName, t1, etc),
      // then places the result in requestBuffer.
      // About placeholders:
      //   %d receive an integers
      //   %s receives a string
      //   %ld receives an unsigned long
      // About sprintf: https://www.programmingelectronics.com/sprintf-arduino/
      sprintf(requestBuffer, requestFormat, dataFileName, t1, te, hu, te2, hu2);

      Serial.println(requestBuffer);

      // Pass the whole request to the client, as a single string.
      client.println(requestBuffer);
      delay(1000);// otherwise the previous request is passed too fast and does nothing on the server (weird)
      
      } else {

        // connection to php file failed
        digitalWrite(11, LOW);//green

      }

      // Flush response, we don't need it.
      while(client.available())
      client.read();
  
      client.stop();
      
   }

}




//**************************************************************************************************************************
// FUNCTIONS
//**************************************************************************************************************************

///////////////////////////////////////////////////////////////
// TIME FUNCTIONS
///////////////////////////////////////////////////////////////

void printTimeElapsed(){
  unsigned long myTime;
  Serial.print(">> Time elapsed since program started : ");
  myTime = millis()/1000;
  Serial.println(myTime); // prints time since program started
  }


///////////////////////////////////////////////////////////////
// WIFI FUNCTIONS
///////////////////////////////////////////////////////////////



void printWiFiData() {
  // print your board's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP address : ");
  Serial.println(ip);

  Serial.print("Subnet mask: ");
  Serial.println((IPAddress)WiFi.subnetMask());

  Serial.print("Gateway IP : ");
  Serial.println((IPAddress)WiFi.gatewayIP());

  // print your MAC address:
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");
  printMacAddress(mac);
}

void printCurrentNet() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print the MAC address of the router you're attached to:
  byte bssid[6];
  WiFi.BSSID(bssid);
  Serial.print("BSSID: ");
  printMacAddress(bssid);
  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI): ");
  Serial.println(rssi);

  // print the encryption type:
  byte encryption = WiFi.encryptionType();
  Serial.print("Encryption Type: ");
  Serial.println(encryption, HEX);
  Serial.println();
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}


///////////////////////////////////////////////////////////////
// LED WAIT FUNCTIONS
///////////////////////////////////////////////////////////////

void blinkLed(int i){
  digitalWrite(i, HIGH);
  delay(100);
  digitalWrite(i, LOW);
  delay(100);
  digitalWrite(i, HIGH);
  delay(100);
  digitalWrite(i, LOW);
  delay(100);
  digitalWrite(i, HIGH);
  delay(100);
  digitalWrite(i, LOW);
  }

Line 250: status = wifi.status(); then print the status
For future projects: please do NOT name something dht2, name it something like sensor_
And add defines for the LED‘s, it is much easier to read „digitalWrite(LED_WIFI_STATUS, HIGH) then digitalWrite(13, HIGH);

I would say add a bunch of debug statements to loop() and, if you're still having problems, post the log file too so others can narrow down where exactly the loop exits.

I think you have a wrap around problem:
https://www.norwegiancreations.com/2018/10/arduino-tutorial-avoiding-the-overflow-issue-when-using-millis-and-micros/

I made an edit to how your millis timer is handled:

millis() returns unsigned long. It is inefficient to keep dividing it by 1000 every loop.
Instead, we convert out intervalTime to milliseconds and compare that way.
Then the only time we need to divide by 1000 is when we send the data to the computer.
unsigned long intervalTime = 5 * 1000; // time between two data collection converted to milliseconds

Also, to avoid overflow erorr, Always structure time realated if() statements as "CurrentTime - StartTime >= TimePeriod"

check out my changes here:
https://gist.github.com/Skuzee/3e16dd72d570a8ac116bef7b95f02771/revisions

I think you have a wrap around problem: https://www.norwegiancreations.com/2018/10/arduino-tutorial-avoiding-the-overflow-issue-when-using-millis-and-micros/

I was thinking the same thing. millis() only overflows every 50 days, so it might not be that. Just in case I restructured the if statement.
I was more worried about dividing my 1000 every loop, so I edited that too.

Another thing to try: Serial.begin(9600); is a pretty slow serial speed. you could increase it all the way to 115200 or more. It miiight be possible if you're sending a ton of serial data every loop that you're filling the serial buffer (which will pause the program until it catches up). Or maybe after a certain point the serial connection is disconnecting and the serial buffer is filling up and halting the program. Two different causes, similar outcomes.

Are there brackets missing?
while(client.available())

Are there brackets missing? while(client.available())

c++ allows a single command below and if/while statement without brackets

// Flush response, we don't need it.
while(client.available())
client.read();

client.stop();

is the same as

// Flush response, we don't need it.
while(client.available()) {
client.read();
}
client.stop();

@frenchguycooking for a pragmatic solution - you could add in a watchdog timer to reset the microcontroller if the program stalls. This is not a fix but it might be all you need to get the data you require.

See https://create.arduino.cc/projecthub/rafitc/what-is-watchdog-timer-fffe20

For your particular board - you might encounter this issue:

https://forum.arduino.cc/t/reset-via-watchdog-for-atmega-4809/689014

Juraj has a solution that could work for the ATMega 4809.

Since the microcontroller might get reset if the watchdog timer is not reset - the init code will run again, meaning a new dataFileName (amongst other side effects).

If it stops working after a consistant number of loops it sounds like you are running out of RAM or space in a buffer (for example serial buffer). Try removing most of your prints and see if it works for more loops. You can wrap your constant strings with F() to store them in flash (similar to harddrive) instaid of ram. Example: Serial.print(F("signal strength (RSSI): "));

// Flush response, we don't need it.
while(client.available())
  client.read();

This is the only while() loop that does not have a serial print. So if it gets stuck in this loop there would be no diagnostics output.
I would suggest:

// Flush response, we don't need it.
unsigned long timeoutTimer = millis();

while(client.available()) {
  if(millis() - timeoutTimer > 60000) { // 60 second timeout
    Serial.println("Flushing Client Response Timed Out..."); // write diagnostic error and break out of loop.
    break;
    }

  client.read();
}
client.stop();

If it stops working after a consistant number of loops it sounds like you are running out of RAM or space in a buffer (for example serial buffer). Try removing most of your prints and see if it works for more loops. You can wrap your constant strings with F() to store them in flash (similar to harddrive) instaid of ram. Example: Serial.print(F("signal strength (RSSI): "));

I too thought it might be a serial buffer issue. However, he is only printing to serial when the data is read. In my experience, when the serial buffer fills up, the program halts at what ever command is trying to write to the serial buffer until the buffer catches up. This can cause hiccups and break time sensitive code, but I would expect that the program would continue eventually. Increasing the serial baud rate isn't a bad idea either way.

I've had an issue in the past where closing the serial terminal on arduino IDE closed the serial connection. Sine the serial buffer was not able to empty then the program halted indefinitely until the serial connection was reconnected.

Edit: Another thing that might help if it is the serial buffer is to add Serial.flush() since that clears incoming buffer and waits for all the outgoing to send.

Are there brackets missing? while(client.available())

c++ allows a single command below and if/while statement without brackets

// Flush response, we don't need it.
while(client.available())
client.read();

client.stop();

is the same as

// Flush response, we don't need it.
while(client.available()) {
client.read();
}
client.stop();

Cool, did not know this, thanks!

Have you tried using another device to make the HTTP requests? Arduinos are great for raw data collection, and can sometimes do more IoT things well, but it seems like there's a memory leak here, and the fastest way to diagnose that is to start delegating features to other devices.

Here's what I would do:

1. Remove all wifi client code
2. Keep the serial printing, especially the stuff generating the HTTP parameters
3. Simplify how the Arduino prints to serial; focus on making it more machine readable. You can still have the status messages, but prefix them with something parseable, like [STATUS], and prefix the data with [DATA].

Use a simple CSV setup for the data, like so:

static char formatted_data[100];
int total_formatted = sprintf(formatted_data, "[DATA] hu1=%d,temp1=%d,hu2=%d,temp2=%d", hu, te, hu2, te2);
if(total_formatted != 4) {
  Serial.println("[STATUS] sprintf call failed, some parameters didn't format correctly");
} else {
  Serial.println(formatted_data);
  memset(formatted_data, 0, sizeof(formatted_data)); // clear it out for next use
}

On the other end you can parse the output of the serial line (happy to help there if you need it)

Good luck on your machine!

Hi Alex,

I think the best way to debug would be to make sure it works without the wireless, monitor the serial output while it's working.
If it goes beyond 300 when plugged in, you know it's a wifi/server issue,
if it doesn't it can be a physical/format issue.

It would be useful to know where the code stop.

If the code stop at 300 loops regardless of the interval, I would suspect a build up of data or connection server. If it was a sensor/board issue, it would be more related to a specific stage.

I noticed in your code that you are declaring the variable in the loop, perhaps it is better to avoid especially since you use long variable. I would be surprise if it is an issue here though.

Perhaps one thing missing is the mitigation of read/writing value. You did it for the server connection, but you directly act on the dht.read() and you directly send the data to the server. Might be useful to check for errors there. For example, before doing a multiplication on dht.read, check if it is not a Nan.

DHT sensor are really not "pasta proof", so you are likely to get weird stuff from them, especially in your setting. Do not even trust the value:
-They can have a long response time in the resting phase (if the air movement is low). I did some test in my lab, some took 10 min to get the value right.
-They are not designed to be used at high humidity (above 80%RH).
-They are not calibrated, which isn't a big issue at normal RH, but bellow 30% or above 80% and you can get wild errors. The calibration is not linear and it's an issue for the "extrema".
-They can have big differences within the same batch, something I observed less with other sensors

Mind you, I am using them, so I think they can be pretty handy, especially to test a system. However, if you find weird value or some inconsistency, keep that in mind!

Last note (not on the code directly), but perhaps it is better if the fan do not blow directly on the pasta. The humidity intake can be much higher near the fan, which can cause a lack of homogeneity in the chamber. it doesn't only affect the pasta but also the sensor.

Good luck!

Hi Alex,

The function blinkLed is blocking your system for 500 milliseconds.
See https://www.arduino.cc/en/Tutorial/BuiltInExamples/BlinkWithoutDelay for an example of a non blocking blink.

Basically it comes down to "don't use delay(nnn)".

The first thing I thought when I saw your Arduino stack in the video was why not use an ESP32, those things are made for WiFi and have a near unlimited memory for an embedded processor.

Then I saw the advice to use ESPHome in the YouTube comments, and I agree, that would be an easy approach because al the connection stuff is handled by the ESPHome software, you only need to configure the thing.
Logging is easiest done with MQTT, InfluxDB and Grafana. ESPHome sends data to MQTT server, Influx stores the data and Grafana visualises the data. See http://nilhcem.com/iot/home-monitoring-with-mqtt-influxdb-grafana.
And if you add a simple relay board, you can switch on/off the dryer automatic.
Don't go for the ESP8266, it's only a bit cheaper as the ESP32 but a lot less powerful.

There are also pre-made ESP32 boards with 4 relays, like the RobotDyn ESP32R4.

If you need any help you may always contact me.
Best regards from the Netherlands

Hi Alex,

You should be careful when using the function sprintf as it can cause memory corruption if you're not careful. Make sure the string you're generating fits within the bounds of the buffer you've provided for the GET request (200 characters). If it doesn't, you'll overflow the buffer and cause memory corruption. Note that using sprintf to record time will always eventually overflow the buffer as the number of digits in the time will always increase past the capacity of the buffer.

For example, suppose I had a buffer of size 4 and only recorded the time. At time t = 10,000 the buffer will overflow since that requires 5 characters to store.

I think 200 characters should be enough space; however, it doesn't hurt to check.

Sincerely,

Emmy Chow from Indiana, USA

@EmDash00 You actually need to have the buffer be one bigger than the biggest possible, for the trailing NULL byte! 200 chars is mostly likely enough though.

@EmDash00 You actually need to have the buffer be one bigger than the biggest possible, for the trailing NULL byte! 200 chars is mostly likely enough though.

Ah right good catch! I've been working in Python too long. You're probably right that the buffer is more than sufficient, but it didn't hurt to check.