Arduino Nano: Send a few bytes over Serial with as little delay as possible

Question

My scenario is the following:

I have a few buttons connected to an Arduino Nano, which is connected to my PC over Serial. Now I want to send a few bytes whenever a button is pressed, to be interpreted by my Computer. However, I noticed a significant, unregular delay (feels like something between almost instantaneous transmission up to a delay of easily ~300 milliseconds), until the message is received by my computer, and I want to get rid of that.

I understand that Arduinos make use of a Serial Buffer, and only send over data once the buffer is full (or some sort of timeout probably), and I think that is the culprit - however I don't see a way of forcing the Arduino to send the data ASAP. I tried just sending over exactly 64 bytes (which, according to what I read, should be the buffer size), but that doesn't help as well (can I however decrease the buffer size or is there a way to send bytes exactly until the buffer is full?)

I don't think baudrate changes anything either, as I'm experiencing problems with the "ping" rather than with the bandwidth (and I also tried anything from 9600 to 115200).

Maybe I'm also completely wrong and the delay happens on the side of my computer, but I don't know if there is anything to change that - I'm looking at the Serial Monitor or at a small pyserial script, both of which have the delay:

arduino = serial.Serial(serial_port}, 115200, timeout=.1)
while True:
    data = arduino.readline().decode("UTF-8").strip()
    if data:
        print(data)

The code on the Arduino-side boils down to:

#include <digitalWriteFast.h>

#define button1_pin 2

int button1_state = 0;
int last_button1_state = 0;

void setup() {
  Serial.begin(115200);
  pinMode(button1_pin, INPUT_PULLUP); 
  while (!Serial) { delay(50); }
  Serial.println("Started");
}

void loop() {
  button1_state = digitalReadFast(button1_pin);
  
  if (button1_state != last_button1_state) {
    if (button1_state == LOW) {
      Serial.write("d\n");
    } else {
      Serial.write("u\n");
    }    
    last_button1_state = button1_state;
    delay(20);
  }
}

Then there's also the weirdest thing which I noticed but am not sure about if it's at all plausible: When the window of the Serial Monitor (the one that ships with the Arduino IDE) is not completely full (meaning there are only some 1-20 rows and the window doesn't need to scroll), I notice the aforementioned delay, but if the Serial Monitor is full (so new content will be on the last line and the window automatically scrolls a bit), I don't. As soon as I clear the output of the window, it's there again. I don't know what to make of this observation, because I definitely don't have any reasonable clue why that may be the case, so maybe it's also just my imagination :shrug:. Using the python-code there is no such behaviour, and the transimission always seems to have a ping of at least 200-300ms.

I hope my problem is understandable, and like I said I am really unsure what even is the culprit (and I see that it's also highly likely that it's on the side of my computer), but does anybody have any idea how I can decrease this delay? Thanks a lot!

Answer 1

"Slowing down to speed up". - a Chinese proverb

The problem is with your python code. Not sure what your computer's operating system is, but if you are using Mac or Linux, if you run top on command line, you will see you have a high CPU utilization causing by your python script. This is because your your python while True loop constantly looping without giving the chances for other processes on your computer to use the CPU, bear in mind that Linux, MacOS or Windows are not designed as a real-time operating system.

So how to solve the problem? put your while True loop to sleep for a short period of time, this allows other processes on your computer to have chance to run and you will see the CPU utilization go down significantly and everything will running smoothly.

import time
arduino = serial.Serial(serial_port}, 115200, timeout=.1)
while True:
    data = arduino.readline().decode("UTF-8").strip()
    if data:
        print(data)
    time.sleep(0.01)

With this slight delay, I've seen some CPU utilisation drop from 90% to 3.5%. This is quite a common mistake that many python programmers made, including some quite experienced programmers who are not familiar with network programming.

Try it out and let me know.

Answer 2

Answering my own question in case anybody experiences the same problem eventually: @hcheung's answer was almost correct after all - see the comments to that question.

Important take-away however: don't trust the Serial Monitor or Python's print to check how quickly the arduino and your computer communicate!

When ponging back to the Arduino to let it trigger an LED, I noticed that the actual transmission is definitely faster than my eye, far from the 200ms delay I mentioned in the question. So the problems for me were

a) the other code I used to test the connection but didn't post here was faulty
b) The delay I saw in the Serial Monitor/python's print is not the delay of the serial communication but apparently somewhere later in the printing of the respective content.

Serial communication is definitely faster than the human eye :)