U-blox GPS module (and TinyGPS library): altitude data getting stuck at strange values

Question

We have built a HAB (high-altitude balloon) tracker based on an Arduino and a U-blox GPS module, but we have a sporadic issue where the altitude data gets stuck at odd (usually low) values. The altitude data of two flights are shown below, showing the issue. All other data (lat, long, time etc) is correct, it's just altitude we have the issue with. We would expect a linear increase in altitude, then a quick drop once it bursts. The squares on the graph mark the data points, so it's not just a case of us missing the data. Sometimes it gets stuck at the current value, and sometimes it just gets stuck at a nonsense value.

For example, in both cases the value is initially stuck at about 270m, which is correct for our launch site. However, it then gets stuck at other nonsense values for a long time (less than 500m) before it starts correctly reporting the altitude. In the 2018 chart you can see that the value drops to 0 for a while.

We have searched online and asked on some forums where people often use the U-blox GPS module, but it doesn't seem to be an issue others have come across. Also, the flights shown here were done on 2 different sets of hardware (Arduino and GPS module), with the 2019 flight using brand new components, so we don't think it's a hardware issue. It's not a transmission issue, as one data set is what was recorded by receivers and uploaded to HABHub, and the other is what was stored on the SD card and recovered after the flight. The same code was used for both, so that is our current suspect. The code had been used in previous years without problem, with only minor modifications between years.

The whole code is pasted below, I have highlighted the main loop with ####### as this is where the reading of the GPS occurs. Some notes on the code:

1. This is the code I've inherited, so I can't necessarily say why things have been done in a certain way.
2. The TinyGPS library looks like it hasn't been updated since this was written (2014ish I think).
3. I am considering re-writing the code from scratch, but I want to make sure I understand the cause of this issue if possible so I can avoid it in the future.

The current hardware is an Elegoo Mega2560 R3 and a u-blox NEO-6M GPS module.

Are we doing something wrong with reading/storing/processing the altitude data?

Edit to add more information and observations:

The code takes time, lat, long, altitude from the GPS module, along with the packet counter, callsign and checksum and assembles them into a string, which is transmitted via radio and stored on the SD card. The string has the following format: $$Callsign,PacketNumber,HHMMSS,lat,long,alt*Checksum

Here's an example packet:

$$XXXXXXXX,1377,121226,51.545547,-1.296546,23*DE62

These packets are generated every 11-12s and stored and transmitted. During the periods of incorrect data, the packets appear fine: counter and time increasing, lat and long giving sensible values, just with a nonsense altitude (in this example, 23m). In the 2018 flight, the altitude is stuck at 267m (the altitude of the launch site) for about 40minutes, even though we have seen it ascend into the clouds. The value then drops to 23m and stays there for over an hour, before briefly jumping to 117m for a couple of minutes before jumping to 15,669m, and from then on reports sensible data (until the drop to 0 during the descent).

What I have just noticed while inspecting the logs is that every time the altitude value jumps to a new nonsense value (either from a sensible value or a nonsense one), the packet that reports the new altitude value is corrupted in some manner! This is true even in the data logged on the SD card, so it is not something that has been introduced by transmitting it, but an issue with generating the string.

Below are 3 adjacent packets received, showing a jump between 2 (nonsense) altitude values:

$$XXXXXXXX,1377,121226,51.545547,-1.296546,23*DE62

$$XXXXXXXX,1379,121226,5.216667,-0.583333,117*CDB2

$$XXXXXXXX,1381,121248,51.547260,-1.293762,117*80C5

Note that the packet counter has increased correctly (by 2, but it's "supposed" to do that, it's not an error), but the timestamp is identical in the first and second packets, and the timestamp of the 3rd is 22s later (they should be every 11s). Also, the second packet (the one with the new altitude), reports a nonsense lat and/or long value. Packets then continue to be sensible (except for the altitude), until at some point it starts reporting correct altitude. Looking though both logs, I can see that this happens every time we get a new nonsense altitude.

Any suggestion as to what is messing up acquiring the altitude data and/or assembling the string? I notice that altitude is handled differently in the code compared to lat/long/time, but I am not aware of the reason for this.

//434.25MHz USB RTTY 50 Baud 500Hz shift ASCII-7 no parity 2 stop bits

#include <TinyGPS.h>
#include <SD.h>
#include <string.h>
#include <util/crc16.h>
#include <Arduino.h>


#define TempAPin A8
#define TempBPin A12
#define PresAPin A10
#define PresBPin A13
#define BatteryVoltagePin A7
#define GreenLED 11
#define RedLED 13
#define RADIOPIN 10
#define RADIOEN 9
int RADIOHIGH = 110;
int RADIOLOW = 100;

int sentance_id = 0;

char UKHAS[80];
char logString[100];
char datastring[80];
char slat[10];
char slon[10];
File myFile; //SD card setup
   
unsigned int TempA = 0;
unsigned int TempB = 0;
unsigned int PresA = 0;
unsigned int PresB = 0;
unsigned int BatteryVoltage = 0;

long lat, lon, alt;
unsigned long date, oldtime, time, fix_age, vel, bearing;
unsigned char dop, sats;
int year;
byte month, day, hour, minute, second, hundredths;

byte gps_set_sucess = 0 ;

TinyGPS gps;

void setup()
{
  delay(1000);
  Serial.begin(9600);
  //Serial2.begin(9600); //Bluetooth
  analogReference(DEFAULT);
  
  pinMode(RedLED, OUTPUT);
  pinMode(GreenLED, OUTPUT);
  digitalWrite(RedLED, HIGH);// Switch both on until GPS module is configured
  digitalWrite(GreenLED, HIGH);
  Serial.println("PROGRAM START");
  SetupGPS();
  digitalWrite(GreenLED, LOW);
  delay(1000);
  
  //SD setup
  //pinMode(10, OUTPUT); //Required for SPI CS to work
  //if (!SD.begin(10)) {
    pinMode(53, OUTPUT); //Required for SPI CS to work
  if (!SD.begin(53)) {
    Serial.println("Card failed, or not present");
    return;
  }
  Serial.println("SD card initialization done.");
  delay(1000);
  //radio setup
   pinMode(RADIOPIN,OUTPUT);
   pinMode(RADIOEN,OUTPUT);
   setPwmFrequency(RADIOPIN, 1);
   digitalWrite(RADIOEN,HIGH);
   Serial.println("FREE RAM:");
   Serial.println(freeRam() );
}


void SetupGPS(void)
{
   // START OUR SERIAL DEBUG PORT
  Serial.println("Initialising GPS....");
  //
  // THE FOLLOWING COMMAND SWITCHES MODULE TO 4800 BAUD
  // THEN SWITCHES THE SOFTWARE SERIAL TO 4,800 BAUD
  //
  Serial3.begin(9600);
  Serial3.print("$PUBX,41,1,0007,0003,4800,0*13\r\n"); 
  delay(2000);
  Serial3.begin(4800);
  //Serial3.flush();
 
  //  THIS COMMAND SETS FLIGHT MODE AND CONFIRMS IT 
  Serial.println("Setting uBlox nav mode: ");
  uint8_t setNav[] = {
    0xB5, 0x62, 0x06, 0x24, 0x24, 0x00, 0xFF, 0xFF, 0x06, 0x03, 0x00, 0x00, 0x00, 0x00, 0x10, 0x27, 0x00, 0x00, 0x05, 0x00, 0xFA, 0x00, 0xFA, 0x00, 0x64, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x16, 0xDC                        };
  while(!gps_set_sucess)
  {
    sendUBX(setNav, sizeof(setNav)/sizeof(uint8_t));
    gps_set_sucess=getUBX_ACK(setNav);
  }
  gps_set_sucess=0;
}

unsigned int charsToInt(unsigned char loByte, unsigned char hiByte)
{
  unsigned int temp = 0;
  temp = loByte + (hiByte<<8);
  return temp;
}

unsigned char lo(unsigned int input)
{
  unsigned char temp = 0;
  temp = (unsigned char)(input & 0xFF);
  return temp;
}

unsigned char hi(unsigned int input)
{
  unsigned char temp = 0;
  temp = (unsigned char)((input>>8) & 0xFF);
  return temp;
}

###################################################################################
void loop()
{ 
  unsigned char b = 0;
  long int counter = 10000;
  int x = 0;
  int mem;
  while(1)
  {
    counter--;
    x = counter % 10000;
    if (x == 1)
    {
      Serial.println(counter);
    }
    
    if (counter==0) 
    {
                
      counter = 100000;
      digitalWrite(GreenLED, LOW);
      digitalWrite(RedLED, HIGH);
      readSensors();
      sprintf(logString, "NOGPS,%i,%lu,0,0,0,%i,%i,%i,%i", sentance_id++, time, TempA, TempB, PresA, PresB);
      Serial.println(logString);
      writeSD();

     sprintf(UKHAS,  "&&XXXXXXXX,%i,NOGPS", sentance_id);
      
      Serial.println(UKHAS);

      writeRADIO();

        
    }
    if(Serial3.available() & 0)
    {
      
       b=Serial3.read();
       
       if (gps.encode(b))
       {
        Serial.println("TEST");
         counter = 1000000;
         digitalWrite(GreenLED, HIGH);
         digitalWrite(RedLED, LOW);
         //Serial.println("Got valid GPS data"); 
         // read values from the gps.
         gps.get_position(&lat, &lon, &fix_age);
         gps.get_datetime(&date, &time, &fix_age);    
         alt = gps.altitude();
         Serial.print("alt=");
           Serial.println(alt);
         vel = gps.speed();
         bearing = gps.course();
         gps.crack_datetime(&year, &month, &day, &hour, &minute, &second, &hundredths, &fix_age);
         readSensors();
         //if (time > (oldtime + 1000))
         //{
           // print the details to BT and SD every 10 seconds   
           oldtime = time;
           // comaptible comms protocol: http://ukhas.org.uk/communication:protocol
       sentance_id++; //count number of packets
           char timeS[10] = {time/100}; 
           char hour[10];
           
           alt = alt / 100; //convert from cm to m
           //if (alt > 50000){ //if nonsense altitude
          //   alt = -1; 
          // }
           double dlat = double(lat);
          //char slat[10];
           dtostrf(dlat/1000000, 8, 6, slat);
           if (lon > 0){ //if west of meridian
             lon = lon * -1;
           }
           double dlon = double(lon);
           //char slon[10];
           dtostrf(dlon/1000000, 8, 6, slon);
           sprintf(UKHAS, "$$XXXXXXXX,%i,%06lu,%s,%s,%lu", sentance_id, time/100, slat, slon, alt);
           //sprintf(UKHAS, "$$XXXXXXXX,%i,%lu:%lu:%lu,%s,%s,%lu", sentance_id, hour, minute, second, slat, slon, alt);
           //change time format
             sprintf(logString, "LOG,%i,%lu,%s,%s,%lu,%i,%i,%i,%i", sentance_id++, time, slat, slon, alt, TempA, TempB, PresA, PresB);
           Serial.println(logString); //debug
           //Serial2.println(logString); //Bluetooth
           Serial.println(UKHAS); //debug
           //50 buad 500Hz
           writeSD();
           writeRADIO();
         //}
       }    
    }//if GPS available  
  }//while(1)  
}//loop
###########################################################################
void writeRADIO()
{
 //digitalWrite(RADIOEN,HIGH);
 //delay(10);
 //Serial.println(UKHAS);

 snprintf(datastring,80,UKHAS); // Puts the text in the datastring

 unsigned int CHECKSUM = gps_CRC16_checksum(datastring); // Calculates the checksum for this datastring

 char checksum_str[8];
 
 sprintf(checksum_str, "*%04X\n", CHECKSUM);

 strcat(datastring,checksum_str);
Serial.println(datastring);
 rtty_txstring (datastring);

Serial.println("MS");
 //digitalWrite(RADIOEN,LOW);
}

void writeSD() //write sensor log to file on SD card
{
// open the file. note that only one file can be open at a time,
// so you have to close this one before opening another.
  myFile = SD.open("log.txt", FILE_WRITE);
  // if the file opened okay, write to it:
  
  if (myFile) {
    Serial.print("Writing to SD card");
    //write full telemetry packet
     myFile.println(logString);

    //close the file:
    myFile.close();
    Serial.println("done."); 
  }
   else {
     //if the file didn't open, print an error:
    Serial.println("error opening SD card file");
  }
}


void readSensors()
{
  TempA = analogRead(TempAPin);
  TempB = analogRead(TempBPin);
  PresA = analogRead(PresAPin);
  PresB = analogRead(PresBPin);
  BatteryVoltage = analogRead(BatteryVoltagePin);
}

void rtty_txstring (char * string)
{
 
/* Simple function to sent a char at a time to
 ** rtty_txbyte function.
 ** NB Each char is one byte (8 Bits)
 */
 
char c;
 
c = *string++;
 
while ( c != '\0')
 {
 rtty_txbyte (c);
 c = *string++;
 }
}
void rtty_txbyte (char c)
{
 /* Simple function to sent each bit of a char to
 ** rtty_txbit function.
 ** NB The bits are sent Least Significant Bit first
 **
 ** All chars should be preceded with a 0 and
 ** proceed with a 1. 0 = Start bit; 1 = Stop bit
 **
 */
 
int i;
 
rtty_txbit (0); // Start bit
 
// Send bits for for char LSB first
 
for (i=0;i<7;i++) // Change this here 7 or 8 for ASCII-7 / ASCII-8
 {
 if (c & 1) rtty_txbit(1);
 
else rtty_txbit(0);
 
c = c >> 1;
 
}
rtty_txbit (1); // Stop bit
rtty_txbit (1); // Stop bit
}
 
void rtty_txbit (int bit)
{
 if (bit)
 {
 // high
 analogWrite(RADIOPIN,115);
 }
 else
 {
 // low
 analogWrite(RADIOPIN,100);
 }
 
// delayMicroseconds(3370); // 300 baud
 delayMicroseconds(10000); // For 50 Baud uncomment this and the line below.
 delayMicroseconds(10150); // You can't do 20150 it just doesn't work as the
 // largest value that will produce an accurate delay is 16383
 // See : http://arduino.cc/en/Reference/DelayMicroseconds
 
}
 
uint16_t gps_CRC16_checksum (char *string)
{
 size_t i;
 uint16_t crc;
 uint8_t c;
 
crc = 0xFFFF;
 
// Calculate checksum ignoring the first two $s
 for (i = 2; i < strlen(string); i++)
 {
 c = string[i];
 crc = _crc_xmodem_update (crc, c);
 }
 
return crc;
}
 
void setPwmFrequency(int pin, int divisor) {
 byte mode;
 if(pin == 5 || pin == 6 || pin == 9 || pin == 10) {
 switch(divisor) {
 case 1:
 mode = 0x01;
 break;
 case 8:
 mode = 0x02;
 break;
 case 64:
 mode = 0x03;
 break;
 case 256:
 mode = 0x04;
 break;
 case 1024:
 mode = 0x05;
 break;
 default:
 return;
 }
 if(pin == 5) {
 TCCR3B = TCCR3B & 0b11111000 | mode;
 }
 else {
 TCCR2B = TCCR2B & 0b11111000 | mode;
 }
 }
 else if(pin == 3 || pin == 11) {
 switch(divisor) {
 case 1:
 mode = 0x01;
 break;
 case 8:
 mode = 0x02;
 break;
 case 32:
 mode = 0x03;
 break;
 case 64:
 mode = 0x04;
 break;
 case 128:
 mode = 0x05;
 break;
 case 256:
 mode = 0x06;
 break;
 case 1024:
 mode = 0x7;
 break;
 default:
 return;
 }
  TCCR3B = TCCR3B & 0b11111000 | mode;
 }
}

// Send a byte array of UBX protocol to the GPS
void sendUBX(uint8_t *MSG, uint8_t len) {
  for(int i=0; i<len; i++) {
    Serial3.write(MSG[i]);
    Serial.print(MSG[i], HEX);
  }
  Serial3.println();
}
 
 
// Calculate expected UBX ACK packet and parse UBX response from GPS
boolean getUBX_ACK(uint8_t *MSG) {
  uint8_t b;
  uint8_t ackByteID = 0;
  uint8_t ackPacket[10];
  unsigned long startTime = millis();
  Serial.print(" * Reading ACK response: ");
 
  // Construct the expected ACK packet    
  ackPacket[0] = 0xB5;  // header
  ackPacket[1] = 0x62;  // header
  ackPacket[2] = 0x05;  // class
  ackPacket[3] = 0x01;  // id
  ackPacket[4] = 0x02;  // length
  ackPacket[5] = 0x00;
  ackPacket[6] = MSG[2];    // ACK class
  ackPacket[7] = MSG[3];    // ACK id
  ackPacket[8] = 0;     // CK_A
  ackPacket[9] = 0;     // CK_B
 
  // Calculate the checksums
  for (uint8_t i=2; i<8; i++) {
    ackPacket[8] = ackPacket[8] + ackPacket[i];
    ackPacket[9] = ackPacket[9] + ackPacket[8];
  }
 
  while (1) {
 
    // Test for success
    if (ackByteID > 9) {
      // All packets in order!
      Serial.println(" (SUCCESS!)");
      return true;
    }
 
    // Timeout if no valid response in 3 seconds
    if (millis() - startTime > 3000) { 
      Serial.println(" (FAILED!)");
      return false;
    }
 
    // Make sure data is available to read
    if (Serial3.available()) {
      b = Serial3.read();
 
      // Check that bytes arrive in sequence as per expected ACK packet
      if (b == ackPacket[ackByteID]) { 
        ackByteID++;
        Serial.print(b, HEX);
      } 
      else {
        ackByteID = 0;  // Reset and look again, invalid order
      }
 
    }
  }
}


int freeRam () {
  extern int __heap_start, *__brkval; 
  int v; 
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval); 
}