1

I'm building a GPS tracker using an arduino nano, Adafruit Ultimate GPS breakout, a SD card slot and a SIM800L module.

So far everything works, the tracker uploads the data to a server as well as saving an extra copy to the SD card. The problem is now, that I don't really think that sending a HTTP request every cycle (gps data point) is very efficient and I would also like to have a way of storing information if there is no GPRS reception.

The idea is to save the GPS data to SD card and bulk send it out every 5 minutes or so.

What would be the best way to set up the data structure? I'm saving the deta in a csv style file:

2016-8-2 22:23:44, 52.53251922, 13.31121666, 18.30, 7, 1.33;

Parsing that is not the problem, but rather how to keep track of what has been sent already and what not. I was thinking about Marking the lines that I sent with a # or something similar as first character.

The only problem I could see with that would be performance, since it could take a while to go though it every time once it's getting a bit larger.

Thanks!

Lukas

1

The only alternative structure I can think of is, since you have a number of data fields to send, you could create a struct to contain all the fields. Write the gathered GPS data to the file in its raw format, with file.write(&your_struct, sizeof(your_struct)) and read in a similar manner into a struct.

  • This will reduce the number of bytes written or read for any data point, and
  • Reduce the overall size of the file, compared to your original method, for the same number of GPS points.

The former improvement means there will be about 3 times as many GPS data points in a single block of data read from or written to the SD card, when compared to your current structure. Another more subtle advantage is that each data point occupies a known number of bytes in the file, making file navigation potentially easier. But you can only use this structure if you don't intend to view the CSV file with some spreadsheet program like MS Excel. You could, of course, very easily write your own PC program to parse the file and write to proper CSV. The Python struct module is very capable.

However, if you wish to continue with your current format, you can keep track of your current position in the file with file.position(); no need to use # as an indicator. You create 2 position variables, one to hold the position of the next line to be uploaded, curr_pos, and the other to hold the position of the end of the file for writing new data, end_pos. Initialize the former to 0.

  • You gather data as usual, and after 5 mins, you save the current file position to end_pos and seek to curr_pos with file.seek().
  • You read a line from the file and send it to the server.
  • If it's sent successfully (use the return code), you update curr_pos with file.position() so that it's now pointing to the next line in the file to be uploaded and go to step 2.
  • Else if the upload fails perhaps due to service, you stop uploading and don't update curr_pos. You seek to end_pos with file.seek() and go to step 1.

You will, of course, use available() to know when you've hit the end of the file to stop uploading, or you can compare curr_pos with end_pos for equality, whichever's more direct for you.

This involves a lot of operations on the file and I/O operations don't come cheap with SD cards, both in terms of current drawn and latency. Depending on how often you collect data and if you don't need to view it on a PC, you might consider some other means of storage (SPI flash, perhaps) or use a more capable Arduino like the Mega with more RAM and store the data temporarily in an array before upload, granting you faster access by orders of magnitude and much reduced power drain in the long term, since each block of file data will be accessed only once.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.