as a follow up to my earlier post Simultaneously reading data from multiple serial ports, I am stuck trying to develop a python sketch that can read the incoming serial data from the com port on my arduino and unpack them from the buffer into two separate csv files, one of the csv files would contain the data from the first sensor while the other would contain the data from the second sensor. I have been able to develop the sketch to a point where on receiving the first header byte the first sensor data is unpacked. I am finding it hard to adjust the sketch to allow for the second byte header to be called to unpack the second sensor data from the data packet. Below is my abbreviated sketch thus far. Thanks all

    def run(self):
        #Creating a csv file to read data from the first sensor
        self.click = open("ball_Data.csv", "wb")
        #Creating a csv file to read data from the second sensor
        self.flick = open("rugby_Data.csv", "wb")
        #creating the serial port object
        self.push = serial.Serial('COM6', 9600)
        while True:
         #If the incoming byte is equivalent to '$'
         if node == b'$':
         # the byte object is equivalent to the available bytes
         node = self.push.read()
         #an array to hold the sensor data
         sensor_array = []
         i = 0
         #This while loop initiates a count from 0 to 9
         #through the sensor array
         while i < 10:
         #The available byte present
         malcolm = self.push.read()
         #The next byte
         turnbull = self.push.read()
         #The incoming uint_16 data is unpacked into the variable lingo
         lingo= (struct.unpack('b', turnbull)[0] << 8) | struct.unpack('B',malcolm)[0]
         #The variable lingo is then appended to the sensor array
         #If the length of the sensor array is equivalent to 9
         #write the data through each count into the sensor array
         if len(sensor_array) == 9:
         self.click.write(str(sensor_array[0] ..+ str(sensor_array[9]))
         #My idea is that this else statement was supposed to trigger
         #the next header for the second sensor which would be an '#'
         #sign before unpacking the data
            oldde = self.push.read()
            sensor_array = []
            i = 10
            while i < 20:
              #The available byte present
              malcolm = self.push.read()
              #The next byte
              turnbull = self.push.read()
              lingo= (struct.unpack('b', turnbull)[10]<<8)|struct.unpack('B',malcolm)[10]
              if len(sensor_array) > 19:
             self.flick.write(str(sensor_array[10]..+ str(sensor_array[19]))

   The Packet data structure for the first and second sensor is:
   Packet1 from sensor 1:
         data_array[0] = headerbyte('#')
         data_array[1] = acceleration_xAxis
         data_array[2] = acceleration_yAxis
         data_array[3] = acceleration_zAxis
         data_array[4] = temperature
         data_array[5] = gyro_xAxis
         data_array[6] = gyro_yAxis
         data_array[7] = gyroz_xAxis
         data_array[8] = checksum
        Packet2 from sensor 2:
         data_array[9] = headerbyte('$')
         data_array[17] = checksum
  • Please fix the code format. The indentation is "off". – Mikael Patel Feb 15 '16 at 9:38
  • @dada He means that it's a snippet and as such there is massive amounts of white space to the left of your listing that causes much of it to be scrolled off the side. You don't need to keep that indenting since it is not in context (which is actually an altogether different issue - we don't deal in snippets here : snippets-r-us.com ) so delete that excess indenting to make it easier to read, or better still, post your entire code so we might have a chance of understanding it. – Majenko Feb 15 '16 at 10:09
  • @MikaelPatel, I adjusted the code format as requested. – dada Feb 15 '16 at 10:26
  • and @majenko , I would comment more on the code to give you a better understanding of what i'm trying to do. – dada Feb 15 '16 at 10:27
  • Can you post an example of the data that is being sent from the Arduino to your computer so that we know what it is that's trying to be parsed? – Majenko Feb 15 '16 at 10:37

I'm somewhat confused by the flow of your program, so instead I will tell you how I would handle the data.

You have a single-byte header, then 7 bytes of data, followed by a single byte checksum.

PySerial's read is (I believe) blocking by default, so it's simple to read the next serial character - unlike an Arduino. It also has a "multi-byte" read version where it can read a pre-set number of bytes from the stream for you.

So my methodology would be:

  1. Read a byte. Is it a '#' or a '$'?
  2. If it is, remember which stream you are dealing with, then:
  3. Read 7 bytes with this.push.read(7) into a byte array (or a string in python < 2.6)
  4. Read one byte for the checksum. If the checksum is correct (whatever calculation you use for that) then unpack the data in the array.
  5. Write the data to the correct file depending on the stream you remembered in step 2.

That way you're not duplicating code for each sensor stream, you are just saying "this packet is stream 2" and using that remembered fact once you have received the packet and verified that it's correct.

A better protocol arrangement, though, which is more reliable, involves both a start byte and an end byte, and your whole packet is wrapped in that pair of bytes. You then have a "sliding window" arrangement whereby the incoming data bytes scroll past a window of a fixed width, and when the first and last characters in that window match your start and end bytes you then extract the middle section and do what you need to do with it. That way if you have a $ or a # that is midway through a stream of data it won't trigger reception since the end byte doesn't match. It makes the whole synchronisation of the data more reliable.

For instance, if you have the current data packet that looks like $ho9*$fup and you miss the first bit of the data for whatever reason (arduino started transmitting before you were ready to receive, for instance) it might trigger reception on the second $ which would give you incorrect values. So instead if you have the start character as $ and the end character as # and include a stream number, so your packet looks like:


Then when you scroll that past the window it would look like:

 $         #
 v         v

And the first and last characters match, so you receive the content.

If you were to miss the first $ the second $ wouldn't match - say the extended stream looks like: $1ho9*$fup#$2hdiw83bf# and you missed $1, the window would look like:

 $         #
 v         v
[$fup#$2hdiw] <= Doesn't match even though it starts with $
[$2hdiw83bf#] <= Matches - starts with $ and ends with #

As you can see the synchronising of your packet is then much more reliable. It would normally have incorrectly triggered on the first $ it saw.

There is always a chance that there may be a $ and a # in just the right positions within successive packets to fool the system, but the chances of that are somewhat slim. You can add extra protection against that by "escaping" the start and stop bytes if they appear within your stream. There's many possible ways of handling that if you really need it.

  • ...Lord God!!! Thanks alot.!!!!!!!!!!!!!!!!!! ...I'm much obliged – dada Feb 15 '16 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.