5

I'm implementing a filter for my IMU Sensor and thus I want close to real time data visualized on the computer. I use binary serial communication to facilitate the sending part for the arduino (as far as I know the serial.print is pretty slow). So I split my int16_t in two bytes and send it, like:

Serial.write((uint8_t)(gx >> 8)); Serial.write((uint8_t)(gx & 0xFF));

After that i directly send the next number (3 in total by now, maybe up to 7 2byte numbers in the future). I read the thing in matlab with:

dt(k) = toc;
tic;
bindata([1:6],k) = fread(s,[6,1],'int8');
time = cumsum(dt(1:k));

Which reads 6 bytes (3 numbers) and then I recalculate the binary representation, concatenate them and get the original number (if someone can suggest an easier way.. I found matlab pretty unhandy here).

The problem is, that the numbers get mixed by the time. Somewhen one byte isn't read or anything, so the bytes get messed up and a nonsense number is produced.One full number (2byte) is skipped for one sample exactly. Instead of this number one number is there twice. The next sample the order is messed up (shifted, such that the first number is second). This process appears after maybe 30 sec, sometimes a few minutes. After the first time it keeps shifting and jumping around.

Can someone tell me, what to do here? Can I include some 'breakpoint'/line terminator, where the reader (matlab) knows, that we are at the start of the first number? Or how is this done actually?

I guess I have to add my main goal: I want to make the sending as fast for the arduino as possible. No extra calculations should be necessary (if possible). And: the reason for these shifts seems to be the some time delay (slowness). I suspect it to be a slow matlab reading, since I saw fluent processing scripts in HIL reading. The errors have stopped however, since I turned the baude rate down. Only wrong numbers are the problem still.

May there be a possibility to loop around the fread and read just store the values after an added 'header'? So let's say a wrong ordering occures. Then I discard everything until the next 'a' char/byte and use the following 6 bytes to produce my 3 values. Then I wait for an 'a' again. For that I would have to loop fread(s,[1,1],'int8'); and search for the header.

Full arduino code:

// Program to send the gyro/accel data via serialport
// corresponding matlab programs: sensing.m and sensing_binary.m
// 2 security loops to guarantee a constant sampling time

// #define DEBUG
#include "GY86.h"
#include "Wire.h"

GY86 gy86;
int16_t ax, ay, az;
int16_t gx, gy, gz;

uint32_t currenttime = 0;
uint32_t starttime = 0;
uint32_t starttime2 = 0;

// #define OUTPUT_ACCEL_COUNTS
#define OUTPUT_GYRO_COUNTS

// #define OUTPUT_ACCEL_BINARY
// #define OUTPUT_GYRO_BINARY


void setup () {
  Serial.begin(9600);
  gy86.setup();
}


void loop () {
  currenttime = millis();
  if (currenttime-starttime > 9)
  {
    while (micros()-starttime2 < 9000) { }
    starttime2 = micros();

    // read raw accel/gyro measurements from device
    gy86.getSensorValues(&ax, &ay, &az, &gx, &gy, &gz);
    // testing constants
    // gx = -29;
    // gy = 245;
    // gz = 17;

#ifdef OUTPUT_GYRO_COUNTS
    Serial.print((int)gx); Serial.print(F("\t"));
    Serial.print((int)gy); Serial.print(F("\t"));
    Serial.print((int)gz); Serial.print(F("\t"));
#endif
#ifdef OUTPUT_ACCEL_COUNTS
    Serial.print(ax);   Serial.print(F("\t"));
    Serial.print(ay);   Serial.print(F("\t"));
    Serial.println(az); Serial.print(F("\t"));
#endif
#if defined(OUTPUT_ACCEL_COUNTS) || defined(OUTPUT_GYRO_COUNTS)
    Serial.print(F("\n"));
#endif

#ifdef OUTPUT_ACCEL_BINARY
    Serial.write((uint8_t)(ax >> 8)); Serial.write((uint8_t)(ax & 0xFF));
    Serial.write((uint8_t)(ay >> 8)); Serial.write((uint8_t)(ay & 0xFF));
    Serial.write((uint8_t)(az >> 8)); Serial.write((uint8_t)(az & 0xFF));
#endif
#ifdef OUTPUT_GYRO_BINARY
    Serial.write((uint8_t)(gx >> 8)); Serial.write((uint8_t)(gx & 0xFF));
    Serial.write((uint8_t)(gy >> 8)); Serial.write((uint8_t)(gy & 0xFF));
    Serial.write((uint8_t)(gz >> 8)); Serial.write((uint8_t)(gz & 0xFF));
#endif

    starttime = currenttime;
  }
}
3

Edit: It seems as the problem is overflow. That means that the USB connection isn't keeping up with the data trying to be sent. To fix this you need to do one of the following things (or both):

  • Higher baud rate. The baud rate is the frequency that data is sent. From what I've heard, anything above 500,000 as a baud rate isn't helpful with the Arduino libraries.
    • The Arduino IDE only goes up so high. Try an application such as PuTTY to get higher baud rates on the serial monitor.
    • Doing very high rates like this are best suited on as short of a cable as you can manage. I'd say 4 feet max, although it depends on many factors, including cable quality. A shorter cable has less resistance (thus less errors).
    • The rest of the answer still applies. You might want to add a simple parity bit to make sure the data doesn't get corrupted while sent. Adding two or three characters greatly reduces the risk of corruption, but at the expense of cutting your sampling rate in half and it doesn't verify data integrity. I don't know your exact situation, so adding a bit might not be possible.
  • Lower sampling rate: you're sending too much, so a simple solution is just to add delay(250); at the end of the loop so you don't overload the port.

Original Answer:

The only thing that comes to mind that is very efficient is a parity bit with another bit that's always the opposite of the parity bit. Why? Having an accidental thing where there the last two bits are opposite of each other and they all add up to a even number (ignoring the last bit) would be really odd.

A parity bit is an extra bit so all the bits added up with the parity bit equals an even number. If it isn't quite right, then you know there's a problem. It works only for odd numbers of bits changed, so it isn't foolproof. An example is you have the bits 10010110. There are four 1s, so it's an even number, thus the parity bit will be 0. If it was an odd number, it would be 1 to make the total count an even number. If the computer calculates it doesn't add up right (excuse my lame pun) then it's corrupted and the computer can discard it.

To implement this you'd need to convert the number(s) and the parity bit to ASCII and then count the 0s and 1s. You can use a remainder function and divide by two so there will be a remainder of 1 if it's odd, thus it's corrupted. I'd personally take the latest bit and the x number before it, and just keep looping until you find a combination that satisfies the whole parity bit thing and is within a reasonable range that you've specified in the code.

A line-break would suffice, but it takes up more bits and discovers only missing data, not corrupt data.

Maybe you should be looking into why there's a problem. Could you reduce the length of the USB cable? Upgrade your cable/try a different one? Slightly reduce the baud rate?

  • The cabling is maybe 2m long. Baude Rate is 115200... what would be the benefit of a lower baude rate? And how do I know how much I need? – mike Jun 24 '14 at 11:00
  • Since I thought the error is in my matlab code, I already opend a SE question for that, since this is now more concerning the delay, rathen than the swaps in bits, let's continue there. Your suggestion works! 1/3s sample time gives valid/current time values! Pls help me to fully understand where this could have come from: stackoverflow.com/questions/24368670/… – mike Jun 24 '14 at 21:08
  • @mike it's not a problem of the Matlab code, it's the Arduino then. You're sending more data than the serial chip can handle at the baud rate you specified. I know I said lower it (I thought info was getting corrupted), but you need to raise the baud rate and lower the sample rate. – Anonymous Penguin Jun 24 '14 at 21:33
  • 25 Hz is possible. That's as high, as possible. However I wanted to debug a quadcopter filtering,... which doesn't quite make that much sense if I run 1/2 or even 1/4 of the planned sampling rate. (I try to get my 100Hz, or at least 50). Also if I add more numbers this phenomenon gets occurs more often I guess? Because it's overflow, right? I'll try to get a shorter cabling, thanks for the advice! – mike Jun 25 '14 at 7:19
  • @mike Glad I could help! Edited my question and I cleaned up the comments here. – Anonymous Penguin Jun 25 '14 at 20:16
3

You could add some kind of packet header/footer to the serial output. For example, if you sent the characters "ab" before the bytes for your number you could reject packets that only had 2 bytes in them. the serial would look like

"ab123ab123ab12ab123"

The matlab code could see another "ab" come in before the previous packet finished and ignore the packet that ended up as only "ab12", and then get itself back on track again. You should use 2 bytes for your header to avoid collisions with the actual data. Adding a checksum at the end of the packet would also allow you to detect bit swap errors. CRC or fletcher16 would work perfectly in this scenario, but in your case a checksum may be more than you need.

  • So I need to add Serial.write('ab') and then write the number bits. And in matlab fread 2+number-bits and check if the first 2 are ab. If not, re-read. Is this the thing you suggest? What do you mean by collision with my data.. If I just read 1+num-bits, and they shift, I would still get an error if not all bits are there, or the first one is not "a"? – mike Jun 24 '14 at 11:04
  • By collision I mean that if your data happened to be the characters 'ab' it would mess up a reading. Your correct about the arduino side, just send "ab" before the data. The matlab code should detect that the sequence "ab" happened and then look for the data bits next, but if another "ab" happens before the data bits are done dump what has already come in and start over. That way it will realign with the data if a byte is dropped. – BrettAM Jun 24 '14 at 17:37
  • Collision you mean the probability, that in my actual values one of the bytes contains the header by incidence? So a longer header would reduce the possibility of having real data = termination string. Correct? – mike Jun 24 '14 at 17:57
  • Exactly. With two bytes there is a 1 in 2^16 probability of random data generating the header. A longer header than 2 bytes that might result in less data making it across since your packets are so small; Sending an extra character per packet would probably cost more time than losing one in 65536 packets. – BrettAM Jun 24 '14 at 18:06
  • Ok fine. I would implement this as Serial.write(219); Serial.write(128); Since what is sent is binary (thus the same as some char), and for matlab it is easier to directly read (&compare) the uint with 8 bit. I mark this as solution, since I can remove the shifts. The whole thing is useless however, since there is some strange time delay that adds up. See: stackoverflow.com/questions/24368670/… – mike Jun 24 '14 at 20:58
0

If what you want is high speed, the serial baud rate is your main concern. I assume that you are using an Arduino with a serial to USB converter chip, such as the Uno or Mega.

Transferring at 115200 baud, you send 11520 bytes/s at normal serial settings (1 start + 8 data + 1 stop bits = 10 bits to send 1 byte of data). With an Arduino running at 16 MHz clock, you can execute 16M / 11520 = 1389 instructions in the time it takes to transmit one byte. So you don't have to worry much about slow library routines, what you have to worry about is how many bytes you send. In that way you are right not to use Serial.print, as it will convert your int to ascii, making all number larger than 99 slower to transmit (for a uint16).

When you send data with Serial.write it is first stored in a ring buffer, then transmitted in the background by an Interrupt Service Routine. The ring buffer is typically 64 bytes. Once the the buffer is full, a call to Serial.write will block until a byte has been transmitted, so there is space in the buffer again. Therefore the timing in your main loop isn't really necessary, you can remove it and your sampling rate will adapt to your serial baud rate.

Once the data has reached the serial to USB converter chip, it will be transmitted on the USB bus at 12 Mbit/s. I don't know the exact data transfer rate since there's a lot more overhead on USB than on the serial, but you can rest assured that the bottleneck isn't here. I don't think you have to worry about USB cable lengths as there is built-in error detection and retransmit in the USB protocol, and the speed is so much faster than the serial link.

In the computer the data can be buffered in several places, it all depends on device drivers and serial libraries used. I'm not an expert here. With no flow control, the drivers/libraries probably have no choice but to drop data if it is receiving more than the end application or later layer can process. Maybe matlab or it's serial implementation or you computer is slow in some way (the data really isn't arriving very fast at all for a modern computer). So you could lower the baud rate or try to deal with lost packets.

I have a suggestion for dealing with lost packets: Reserve the upper x bits in each byte you transmit to hold the number of that byte in its sequence. You could f.ex. reserve 2 bits, allowing you to send up to 4 byte packets belonging together. Each byte-packet would have 6 data bits, enabling you to send numbers with 24 bits of precision. To send a uint16:

void send_int16(uint16_t data)
{
  Serial.write((0 << 6) | (data & 0b00111111));
  data >>= 6;
  Serial.write((1 << 6) | (data & 0b00111111));
  data >>= 6;
  Serial.write((2 << 6) | (data & 0b00111111));
}

Then in matlab you should wait for a sequence of packets numbered 0, 1, 2 in the two uppermost bits, remove the numbering and put the packets together with bitwise operators. If you get a malformed sequence, eg. 0, 2, can discard it and wait for the next byte starting with a zero.

I haven't really used matlab, so you are on your own.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.