Arduino sends bad data ONLY when not connected at the launch my application

Question

I'm trying to write a little sensor reading application with a light sensor attached to an Arduino, which writes the information from the sensor over serial. I have written a python program to read the data from the serial port. The good news, is that my program is working completely fine... only if I make sure that the Arduino USB is connected to my laptop before I launch my Python program.

I have myself forgotten to connect the Arduino before running the program before, which can leave it hanging... until you realise and plug the Arduino back in. It would be nice to have a window helping users if they ever make the same mistake.

Here is the Arduino sketch I have running on my Arduino.

int photoTran = A1; 
  int reading = 0;

void setup() {
  pinMode(photoTran, INPUT);
  Serial.begin(9600); 
}

void loop() {
  // We just read the value on the analogPin
  reading = analogRead(photoTran); 
  Serial.println(reading); 
  delay(100); 

}  

Here is the little catcher part of the Python script which checks whether the Arduino is connected before I get into the main part of my application. When I plug in the Arduino before running the Python script, it can still sometimes take a few attempts before connecting. (Just FYI: the window(), button, values, variables are from the PySimpleGUI framework that I'm using.)

connected = False
while connected == False:
    button, values = window.Read(timeout=300)
    if button in (None, 'Exit'):
        break
    try:
        print("Attempting to connect")
        SER = serial.Serial(serial_port, baud_rate, timeout=0.1)
        connected = True
        ser.append(SER)
    except:
        time.sleep(0.1)
        attempts += 1
        window.FindElement('attempts').Update(attempts)
window.close()

# User exited from the GUI connection window
if not connected:
    sys.exit(0)

Later on in my Python code, I want to read the sensor data coming to us from the Arduino. ReadArduino, and CleanThe are the functions that I use to read data from the Arduino.

def ReadArduino( serialCon ):
""" Try to read in a value from the Arduino. Sometimes we might read a value from the serial
    port before the arduino has sent anything. In such cases we will wait a small time, and read again.
    Until we pick up our first non empty integer value."""
while True:
    val = serialCon.readline()
    print(val)
    val = cleanThe(str(val))
    if val != '':
        return val

def cleanThe( lineVal ):
""" The Arduino returns non-clean data back to us.
    We need to remove all the non-integer values to get
    a nice plottable integer value. Will accept '.' unconditionally for now,
    we may have to add conditions later. """
S = {'1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '.'}
clean = ''
for char in lineVal:
    if char in S:
        clean += char
return clean

Now the confusing part. When I run the application without the Arduino connected to my computer. It gets stuck in the while connected == False: while loop above. This is exactly what I want. Then when I plug it in, the serial port sets up and we break out of the loop. I even have a serial port object that I read from.

However, when I read from the serial port, I get back garbage hex values, without any sign of any useful integer values that I would expect.

Here are some sample raw val values that get read back when the python script is run with the Arduino connected before running: b'336\r\n' b'349\r\n' b'343\r\n' b'322\r\n'

Then here is the output I receive when the Arduino was not connected to my computer before running the Python script.

b'\xe0\x00\xe0\x00\x00\xe0\x00\x00\x00\x00\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\xe0\x00\x00\x00\x00\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\xe0\x00\x00\xe0\x00\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\x00\x00\x00\xe0\x00\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\x00\x00\x00\xe0\xe0\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\x00\x00\x00\x00\xe0\x00\xe0\xe0\x00\x00\xe0\x00\xe0\x00\x00\x00\x00\x00\xe0\x00\x00\xe0\xe0\x00\x00\xe0\x00'

b'\x00\x00\x00\xe0\x00\x00\xe0\x00\x00\xe0'

I have noticed, that when I am running my script with the Arduino plugged in from the start (i.e. the working case). The first value I read back from the Arduino is the same type of garbage hex:

b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

It seems like in the working case, the hex is all sent on one line. Where as in the non-working case, these hex values, seem to be sent intermittently in smaller chuncks. Perhaps I could add a check to the Arduino code, to only write values back to serial if they are good? I do realise this may be more of a python serial issue than an Arduino issue, let me know if this is the case and I will move the post.

Has anyone experience some kind of problem like this before?

Answer 1

Just to explicitly show what I changed to my python code I thought I would answer my own question. The solution I found ended up being in the Python, but the other suggestions were useful for cleaning up the output of my Arduino. Here is the old auto connecting solution.

connected = False
window = sg.Window('Arduino Connection').Layout(layout)
attempts = 0
while connected == False:
    print("In connection while loop")
    button, values = window.Read(timeout=300)
    if button in (None, 'Exit'):
        break

    try:
        print("Attempting to connect")
        SER = serial.Serial(serial_port, baud_rate, timeout=0.02)
        print(SER)
        connected = True
        ser.append(SER)
    except:
        time.sleep(0.1)
        attempts += 1
        window.FindElement('attempts').Update(attempts)

Here is the working solution where I ask the user to press a connect button.

    connected = False
window = sg.Window('Arduino Connection').Layout(layout)
attempts = 0
while connected == False:
    print("In connection while loop")
    button, values = window.Read(timeout=300)
    if button in (None, 'Exit'):
        break
    if button == 'Ok':
        try:
            print("Attempting to connect")
            SER = serial.Serial(serial_port, baud_rate, timeout=0.02)
            print(SER)
            connected = True
            ser.append(SER)
        except:
            time.sleep(0.1)
            attempts += 1
            window.FindElement('attempts').Update(attempts)

Answer 2

On the Arduino side make your code non blocking:

 unsigned long timeStamp = 0;
 unsigned long timerDelay = 1000; // 1 second

 setup(){....}

void loop() {
// We just read the value on the analogPin
 if (millis() - timestamp > timerDelay ) {
     reading = analogRead(photoTran); 
     Serial.print(reading); // instead of println which sends additional \r\n
     timeStamp =millis();  // reset timer instead of delay(100); 
 }
}

On the Phyton side try to implement a routine waiting for the first char and then start reading from the next message on. The 1 second between sends on Arduino should suffice to ensure its working. The garbagge chars could be an init sequence from the port driver - this can not come from your code,