arduino due-Python interfacing for high speed data rate

Question

Four our project, I want to send data mesured by an Arduino due to a computer via Pyton very vast. So I tried to make a simple benchmark by sending 10240 int datas to determine the time it take. In order to do this, I first try using the programming port and execute this simple script:

int i=1;
int maxv=10240;
int value[10240];
int deb, fin;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(250000); //serial initialisation
  for (i=0 ; i<maxv ; i++){
    value[i]=i;
  }
  delay(200);
}

void loop() {
  // put your main code here, to run repeatedly:
  Serial.println("starti"); //flag
  deb=micros();
  for (i=0 ; i<maxv ; i++) { // sending loop
      Serial.println(value[i]);
  }
  fin=micros();
  Serial.println("stopi");
  Serial.println((fin-deb)/1000000.0);
  delay(2000); //waiting  
}

So with this I can mesure the time it take to send all the datas. And I receive it with this Python's script:

import numpy as np
import serial
ser = serial.Serial('/dev/ttyACM0', 250000)

value=np.empty(10240)

temp=ser.readline()

while temp!='starti\r\n': #search for the flag
    temp=ser.readline()

for j in range(10240):
    value[j]=int(ser.readline())

ser.readline()

time=float(ser.readline())
print time

with this beaudrate, it take nearly 2.4s which too slow for my purpose. If try another baudrate, 500000 for example I read data like this : �Ë$Pd@s`¥š... I have this for all baudrate higher than 250000.

So I tried the native port which normaly send the data as fast as he can, but we this it took nearly 1.3s, and it's again too slow for my purpose.

I checked that it is not python wich is too slow, by doing the same than in the last example:

import time
import numpy as np

value=np.empty(10240)

deb1 = time.clock()

for j in range(10240):
    value[j]=j

fin1= time.clock()
print fin1-deb1

and it took only 4ms. So is it clearly the arduino wich send the data to slowly.

So how can I increase the data flux? I have seen the serial.write(buf,len) function which seems does not convert the data into ascii caracters. Does this function will be faster by putting the correct arguments? And, if it's the cas, how can I read this with Python?

Answer 1

There's two things you need to know:

1. The Serial.write(buf, len) function sends the raw data - one byte for every "len" count.
2. On the Due integers are 32-bits, so each one takes 4 bytes.

So first you need to define how big the numbers are you'll be dealing with and use data types that are big enough without being wasteful - you have 3 choices: char, short and int - 8, 16 and 32 bits respectively.

• Values between 0 and 255 use an unsigned char (uint8_t)
• Values between -128 and 127 use a signed char (int8_t)
• Values between 0 and 65535 use an unsigned short (uint16_t)
• Values between -32768 and 32767 use a signed short (int16_t)
• Values between 0 and 4,294,967,295 use an unsigned int (uint32_t)
• Values between −2,147,483,648 and 2,147,483,647 a signed int (int32_t)

One very tricky thing with this kind of binary data transfer is knowing where a value starts and ends. You don't get that kind of issue with ASCII representations as you have a line feed and/or a carriage return after each full value and the range of valid characters that make up a value is very narrow and can never contain the delimiting character. You don't have that luxury with binary transfers as there is no byte value that can't appear in a data stream. However, there could be byte sequences that could never appear in the data stream - numbers you know you'll never be using that could form part of a synchronizing header (say, in hexadecimal, 0xFF 0xFF 0xFF 0xFF 0x00 0x00 0x00 0x00 - or in signed shorts that would be a sequence of 65535 65535 0 0) - once you see that sequence of bytes you know you're lined up with your data stream and can receive it byte by byte and build up the right sized values from that.

You can estimate your transfer time by working with the baud rate and knowing the data size. For instance, to transfer 1000 short (16 bit) values at 250000 baud should take around:

• 1000 * 2 = 2000 bytes
• 250000 baud = 25000 bytes per second (8 bits for the byte, 2 bits for the start and stop bit)
• 2000 / 25000 = 0.08s