0

Happy New Year!

So I have made a Nano send 4 pot values to an Arduino Uno over serial but I have a problem: I don't know how to separate these values. Here's the code:

Master:

    int Pot = A0;
    int Pot2 = A1;
    int Pot3 = A2;
    int Pot4 = A3;
    
    void setup() {
      Serial.begin(115200);
    }
    
    void loop() {
      int Value = analogRead(Pot);
      int Value2 = analogRead(Pot2);
      int Value3 = analogRead(Pot3);
      int Value4 = analogRead(Pot4);
      Serial.write(Value); //Write the serial data
      Serial.write(Value2); //Write the serial data
      Serial.write(Value3); //Write the serial data
      Serial.write(Value4); //Write the serial data
      delay(150);
    }

Slave:

    int incomingByte; // for incoming serial data
    
    void setup() {
      Serial.begin(115200); // opens serial port, sets data rate to 9600 bps
    }
    
    void loop() {
      // send data only when you receive data:
      if (Serial.available() > 0) {
        // read the incoming byte:
        incomingByte = Serial.read();
    
        // say what you got:
        Serial.print("I received: ");
        Serial.println(incomingByte, DEC);
      }
    }

And a bit from the serial logs:

    I received: 255
    I received: 247
    I received: 11
    I received: 0

It updates and sends these four values for the four pots, it can read them but they are together, and it sometimes also kind of lags if I can say it like that; it sometimes takes longer to update but then it goes to normal again.

Thanks!

1

This is actually a huge topic, one that I could write volumes on. However I will try and be concise. But it is such a common question it's time to finally write a definitive answer.

Sending data between devices, be that computers, Arduinos, or anything else like that, is very much like speaking to someone.

It is very common to do something like you have done and just "send" the data. However that is, as you have rightly noticed, very much akin to sending someone a message all in capitals with no spaces or punctuation. Like this:

WHATAPIECEOFWORKISMANHOWNOBLEINREASONHOWINFINITEINFACULTYINFORMANDMOVINGHOWEXPRESSANDADMIRABLEINACTIONHOWLIKEANANGELINAPPREHENSIONHOWLIKEAGODTHEBEAUTYOFTHEWORLDTHEPARAGONOFANIMALSANDYETTOMEWHATISTHISQUINTESSENCEOFDUSTMANDELIGHTSNOTMENONORWOMANEITHERTHOUGHBYYOURSMILINGYOUSEEMTOSAYSO

Only through effort and the application of known predefined standards (i.e., English vocabulary) is it possible to make out what the message says.

So in language we introduce "meta" information - that is, information in a message that tells you not about the content of the message, but about the message itself. The simplest is to introduce spaces to indicate where words start and end. So that message now becomes:

WHAT A PIECE OF WORK IS MAN HOW NOBLE IN REASON HOW INFINITE IN FACULTY IN FORM AND MOVING HOW EXPRESS AND ADMIRABLE IN ACTION HOW LIKE AN ANGEL IN APPREHENSION HOW LIKE A GOD THE BEAUTY OF THE WORLD THE PARAGON OF ANIMALS AND YET TO ME WHAT IS THIS QUINTESSENCE OF DUST MAN DELIGHTS NOT ME NO NOR WOMAN EITHER THOUGH BY YOUR SMILING YOU SEEM TO SAY SO

There, that's much more readable. But still lacking. So we add more "meta" characters that provide even more information about the formation of the message: punctuation. Now we can have:

What a piece of work is a man! How noble in reason, how infinite in faculty! In form and moving how express and admirable! In action how like an Angel! in apprehension how like a god! The beauty of the world! The paragon of animals! And yet to me, what is this quintessence of dust? Man delights not me; no, nor Woman neither; though by your smiling you seem to say so.

And now the message is completely plain. What we have done here is introduce characters that aren't part of the message but form the structure of the message. These characters are distinct from the message by being of a different type of character. Not letters, but symbols instead. It's possible to add any symbol you could imagine and attach a meaning to it to convey any information about the message you like. There is no limit.

But that's where computer communication differs.

When sending data over serial you have a limited range of values you can send. You have one byte for sending information, which means that you can only send one of 256 possible "symbols".

When you're sending, as you are, binary data you use all 256 symbols for your data. There are no symbols left for any "punctuation" in your message. It becomes impossible to go even from the first to the second examples above, let alone the third.

So what can be done?

Well, there's two approaches:

  1. Re-encode your data to use less "symbols" leaving some available for meta information
  2. Nominate some of the symbols used for data to be also used for meta information and create some way of distinguishing them from the actual data

The first method is probably the simplest - change the data encoding to use a smaller range of symbols. The most obvious way is to use ASCII encoding - use just 10 symbols to represent numerical data, but use a lot more of them. I'm sure you know the symbols: "0123456789". You then have such standard symbols as "New Line" (or \n) to mark when your message is finished, and commas to separate different parts of your message.

It's easy to encode and transmit, but harder to receive. Sending is as simple as:

Serial.print(Value);
Serial.print(F(","));
Serial.print(Value2);
Serial.print(F(","));
Serial.print(Value3);
Serial.print(F(","));
Serial.println(Value4);

Receiving that data and making sense of it is far more involved. You have to read everything up to a comma (or to the end of the line and split it on a comma) and then convert the ASCII text back into numbers. It could be done on the fly on a character-by-character bases (read a character, if it's a number then multiply your current value by 10 and add the new number to it, if it's a comma then move on to to the next number, or if it's the end-of-line then reset to the first number again) or as a whole chunk of data (read the whole lot into a char array until you reach \n then split it with strtok() on the commas, and convert each chunk with atoi()).

That method though is not efficient. It takes a lot of bytes to represent just a small number (for instance the number 100 takes 3 bytes whereas the value 100 will fit into a single byte as a raw value), and processing it afterwards is slow and cumbersome.

Method 2 from the list above is far more efficient, but takes a little more in-depth knowledge and thought.

Step one is to design a strict format for your message. You need some way of saying "This is the start of the message" and "This is the end of the message" and everything in between is considered the data for the message. Since you are working with fixed sized variables you don't need anything within that data to separate the values.

You have 4 integer values. That's 8 bytes in total (each integer takes 2 bytes for 16 bits). You could pack those together in to a struct for easier handling:

struct data {
    int value1;
    int value2;
    int value3;
    int value4;
};

struct data values;

That is good if you have lots of different kinds of data, but since yours are all the same a simple array might be better:

int values[4];

Now the ASCII character set has a set of control characters that are intended to be used for building these kind of messages. For example character 2 is "STX" or "Start Of Text". Character 4 is "EOT" or "End Of Transmission". So you could choose to start your message with character 2, then send your 8 bytes, and follow it with character 4. Then you just look for a 2, read 8 bytes, and check it's followed by a 4. That would work, right?

Well, no. Not quite. What if you're sending the value 2 or 4 as part of your data? How would it distinguish those from the start/end characters? You can't.

It's now time to introduce a new concept: Data Link Escape. This is where you nominate another special character. This one says "The next character is data. Treat it as such". DLE is character 16 in ASCII, so you might like to use that. 27 is sometimes used too, or 255. You can use whatever you like, but I like 16 (DLE).

The idea here is that if you are sending a byte that could be either a control character or a data character (i.e., 2 or 4) then you first send the DLE character if and only if that character should be interpreted as data. So if you are sending a start byte you send just 2. If you're sending the value of 2 in your data you send 16 then 2.

But what, you ask, if you are sending the value 16 in your data? Simple: you just send it twice. The first time is the DLE character which says the next byte is data. The next is interpreted as data regardless because it's following the DLE character.

Sounds complex, yes? Well, not really. Let's visualise it. Take our original text message. Let's nominate A as the start of message and B as the end of message. We'll even add C as a space character. D will be our DLE character. Our message would now look like (I won't do it all, too tedious...):

AWHDATCDACPIEDCECOFCWORKCISCMDANB

To you as a human it looks garbled, but that's because you're trying to apply standard English rules to it. But if we take it one character at a time and apply our new rules to it it's different:

  • A - start of message
  • W - letter W
  • H - letter H
  • D - Next character is a letter
  • A - letter A
  • T - letter T
  • C - Space

... etc ...

We do just the same with your binary data. Let's say we're sending the values 23, 2, 16 and 1928 as integers. In little-endian (I'll go into that later on) that's represented as the values

23,0,2,0,16,0,136,7

so with our start and end characters, and adding our DLE character 16 we end up with

2,23,0,16,2,0,16,16,0,136,7,4

You might like to write a little function to send data in that format:


int values[4];

// Some symbols for easy reference
#define STX 2
#define EOT 4
#define DLE 16

void sendPacket(const uint8_t *data, int len) {
    Serial.write(STX);
    for (int i = 0; i < len; i++) {
        // Escape the special characters
        if ((data[i] == STX) || (data[i] == EOT) || (data[i] == DLE)) {
            Serial.write(DLE);
        }
        Serial.write(data[i]);
    }
    Serial.write(EOT);
}

// then use it:

sendPacket((uint8_t *)values, sizeof(values));

Now reception can be done one byte at a time and fed into your array as raw data. Something like this:


int values[4];

// Some symbols for easy reference
#define STX 2
#define EOT 4
#define DLE 16

bool rxData(uint8_t *data, int len) {
    static int pos = 0; // Where in the data are we now?
    static bool isDLE = false; // Are we in DLE mode?
    static bool messageStarted = false; // Are we receiving yet?

    if (Serial.available()) {
        int b = Serial.read(); // Read one byte

        if (!messageStarted) { // We're not receiving yet - should we start?
            if (b == STX) { // STX received, start receiving.
                messageStarted = true;
            }
            // Anything else is ignored.
            return false; // Message not complete yet
        }

        // If we're here we must be part way through receiving.

        // If we received a DLE last time then we always treat it as data
        if (isDLE) {
            data[pos++] = b;
            isDLE = false; // No longer in DLE mode
            if (pos > len) { // Too many bytes in the message, something bad happened
                // ABORT
                pos = 0;
                messageStarted = false;
            }
            return false; // Message not complete yet
        } 


        // If we have an STX and we're not in DLE mode then we must be
        // out of sync. Reset the counter and flags and start a fresh message
        if (b == STX) {
            pos = 0;
            isDLE = false;
            return false; // Message not complete yet
        } 

        // Are we at the end of the message?
        if (b == EOT) {
            if (pos == len) { // Did we receive the right number of bytes?
                messageStarted = false;
                return true; // Message complete
            }
            // Something bad must have happened. Not the right amount of
            // bytes received for a valid message. ABORT.
            messageStarted = false;
            pos = 0;
            isDLE = false;
            return false; // Message not complete yet
        }

        // Check for a DLE character
        if (b == DLE) {
            isDLE = true;
            return false; // Message not complete yet
        }

        // Anything else is a data byte.
        data[pos++] = b;
        if (pos > len) { // Too many bytes in the message, something bad happened
            // ABORT
            pos = 0;
            messageStarted = false;
        }
    }
    return false; // Message not complete yet
}

// Then use it:

if (rxData((uint8_t *)values, sizeof(values)) {
    Serial.print("You got: ");
    Serial.print(values[0]); 
    Serial.print(", ");
    Serial.print(values[1]); 
    Serial.print(", ");
    Serial.print(values[2]); 
    Serial.print(", ");
    Serial.println(values[3]); 
}

Now I haven't tested any of this code, so take it with a pinch of salt.

Now a little word on endianness. An integer is 16 bits in size. That takes two bytes to represent (2 x 8 = 16). The "endianness" of a system is the order in which those two bytes are stored in memory. In a little-endian system, like the Arduino, the value 1928 is stored as the bytes 136,7. But on a big-endian system it's stored as 7,136. The "biggest" (i.e., most significant byte) value comes first. When sending binary data like this it's vital to ensure that both ends of the link agree on the order of the bytes. And then there's word size to consider. 8-bit systems use a 16-bit word for integers. 32-bit systems use a 32-bit word. So sending an int from an Arduino to a Pi, for example, will cause confusion - the Arduino sends 2 bytes (for 16 bits) and the Pi expects 4 bytes (for 32 bits).

To combat the latter it is good to use the fixed width variable types that specify exactly how many bits are used. Instead of int use uint16_t (for Unsigned INTeger 16 bits). That way you're unambiguous about what size it is.

To combat the former problem (if needed) there are a standard set of functions that alter values from native to "network" endianness. Network endianness is a standard used for IP communication and is the same as big-endian. If you're only ever going to communicate between little-endian Arduinos then there's no real need for this. But if that is not going to be a certainty then using htons() for "Host to Network Short" and ntohs() for "Network to Host Short" ("short" is a 16 bit word, like int16_t) could be of benefit. It's best to use them when storing or retrieving values in your values[] array:

values[0] = htons(analogRead(0));

and:

Serial.print(ntohs(values[0]);

You may need to include the header machine/endian.h in your code depending on which system you're using.

I could go into so much more depth about things like variable length packets, checksums, sliding window reception and such, but for now this covers the basic concepts. Maybe I'll write a book.

1
  • Thank you. I never read an answer like this. Please write the book, I'll buy it whatever it may take. – ridgy Jan 4 at 21:00
3

If your goal is to "talk" to a computer vs. a human, then a format that can be parsed easily is better than one that is easy to read.

For example, you have:

I received: 255
I received: 247
I received: 11
I received: 0

While this could be parsed by your Uno, it will be a lot simpler if you send the data something like:

255|247|11|0

Now you can use a "C" string parsing function like strtok() to parse on the "|" character. Of course there are many other ways to do this as well but this is one possible solution to your question.

0

So like this? And also what kind of serial speeds do I use? @Majenko This is the sender:

    int pot1 = A0;
    int pot2 = A1;
    int pot3 = A2;
    int pot4 = A3;
    
    
    
    // Some symbols for easy reference
    #define STX 2
    #define EOT 4
    #define DLE 16
    
    
    void setup() {
    Serial.begin(9600);
    }
    
    
    void sendPacket(const uint8_t *data, int len) {
      Serial.write(STX);
      for (int i = 0; i < len; i++) {
        // Escape the special characters
        if ((data[i] == STX) || (data[i] == EOT) || (data[i] == DLE)) {
          Serial.write(DLE);
        }
        Serial.write(data[i]);
      }
      Serial.write(EOT);
    }
    
    
    void loop() {
    
      int values[4] = {analogRead(pot1), analogRead(pot2), analogRead(pot3), analogRead(pot4)};
      sendPacket((uint8_t *)values, sizeof(values));
      
    
    }

And this is the receiver:

int values[4];

// Some symbols for easy reference
#define STX 2
#define EOT 4
#define DLE 16


void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}


bool rxData(uint8_t *data, int len) {
  static int pos = 0; // Where in the data are we now?
  static bool isDLE = false; // Are we in DLE mode?
  static bool messageStarted = false; // Are we receiving yet?

  if (Serial.available()) {
    int b = Serial.read(); // Read one byte

    if (!messageStarted) { // We're not receiving yet - should we start?
      if (b == STX) { // STX received, start receiving.
        messageStarted = true;
      }
      // Anything else is ignored.
      return false; // Message not complete yet
    }

    // If we're here we must be part way through receiving.

    // If we received a DLE last time then we always treat it as data
    if (isDLE) {
      data[pos++] = b;
      isDLE = false; // No longer in DLE mode
      if (pos > len) { // Too many bytes in the message, something bad happened
        // ABORT
        pos = 0;
        messageStarted = false;
      }
      return false; // Message not complete yet
    }


    // If we have an STX and we're not in DLE mode then we must be
    // out of sync. Reset the counter and flags and start a fresh message
    if (b == STX) {
      pos = 0;
      isDLE = false;
      return false; // Message not complete yet
    }

    // Are we at the end of the message?
    if (b == EOT) {
      if (pos == len) { // Did we receive the right number of bytes?
        messageStarted = false;
        return true; // Message complete
      }
      // Something bad must have happened. Not the right amount of
      // bytes received for a valid message. ABORT.
      messageStarted = false;
      pos = 0;
      isDLE = false;
      return false; // Message not complete yet
    }

    // Check for a DLE character
    if (b == DLE) {
      isDLE = true;
      return false; // Message not complete yet
    }

    // Anything else is a data byte.
    data[pos++] = b;
    if (pos > len) { // Too many bytes in the message, something bad happened
      // ABORT
      pos = 0;
      messageStarted = false;
    }
  }
  return false; // Message not complete yet
}


void loop() {
  // put your main code here, to run repeatedly:
  if (rxData((uint8_t *)values, sizeof(values))) {
  Serial.print("You got: ");
    Serial.print(values[0]);
    Serial.print(", ");
    Serial.print(values[1]);
    Serial.print(", ");
    Serial.print(values[2]);
    Serial.print(", ");
    Serial.println(values[3]);
  }
}
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.