0

I'm working on a project that reads vibration on a building using an accelerometer and displaying it's data on a web server. The plan is to read 10 samples/second from the accelerometer and send that data every 5 seconds to a web server using arduino yun.

So there would be 10(samples/sec) x 5(seconds) x 3(3 axis - x,y,z) = 150 data points that will be sent to the webserver, I tried this with the client.get() command but it cannot send that much data to a web server, I also tried POST request but i've had trouble with the sending it multiple times since it needs to have a delay.

What are the other options that I can do in order to achieve this? Any suggestion would help and please if you think my question is bad or not stated properly don't report me, I'm still new here in stack exchange. Any help/suggestion is much appreciated

Arduino Code for POST request

#include <Bridge.h>
#include <HttpClient.h>
#include <YunClient.h>

//IPAddress server(192,168,1,200);
byte php_server[] = {192,168,1,9};

YunClient client;
String data = "", datax="", datay="", dataz="";
unsigned long ts_0 = 0, tr_0 = 0, t_read = 100;
int counter=0;

void setup() {
  // put your setup code here, to run once:
  Bridge.begin();
  Serial.begin(115200);

}

void loop() {
  // put your main code here, to run repeatedly:

  if((millis()-tr_0>=t_read)) {

    if(counter<49) {
      counter++;
    }
    else if(counter == 49) {
      //Serial.print("sent");
      String dat = "0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12,0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12, 0.24,1.24,4.56,8.97,3.15,7.23,9.34,0.12,6.21,8.12";

      send_to_ws(dat);
      Serial.println(dat);

      counter = 0;
    }
    tr_0=millis();
  }
}

void send_to_ws(String data) {
  if(client.connect(php_server, 80)) {
    Serial.println("Connected");        
    client.println("POST /arduino/add.php HTTP/1.1");
    client.print("Content-Length:");
    client.println(data.length());
    Serial.println(data.length());
    Serial.println(data);
    client.println("Connection: Close");
    client.println("Host:192.168.1.9");
    client.println("Content-Type: application/x-www-form-urlencoded");
    client.println();
    client.println(data);
  } else {
    Serial.println("Connection error");
  }
}

Serial monitor output when running this sketch, It prints it in the serial monitor but only blank data is being pushed in the database of the server here

  • Its not clear from your question, are you sending 150 POST requests or one request with 150 points of data? What are the minimum and maximum values of the points and who many decimal places are you using? – Code Gorilla Dec 1 '16 at 13:00
  • one request with 150 points of data, i think the maximum value wouldn't exceed the 10 so probably between +/- (0.00 - 9.99) considering i will have 2 decimal places, – Ralph Dec 1 '16 at 13:12
  • 1
    Could you show us your code? As in principle both method would work. There is probably something in the code, like e.g. a buffer, that limits the amount of data that can be send. – Gerben Dec 1 '16 at 13:33
  • @Gerben I haven't really integrated all the things i wan't to do in a sketch, but what i did was sending a request and hardcoded the data that i want to send. I found out that it is too large especially using the client.get() method of the httpclient library since sending a small string can be done without a problem but when i try to increase the size (up to 60 data points) it can still send but it takes so much time sending the data, approximately 20 sec and when I'm using POST request which uses delay i dont know how i can still read every 100 ms while having a delay for 1-2 sec in sending – Ralph Dec 1 '16 at 14:11
  • Could you show us your code? I don't care that it isn't finished. It's even better like that, as there is no other code that can have side effects. Why does POST use delays? That makes no sense. – Gerben Dec 1 '16 at 16:08
1

Option 1. Send a single message from the client to the server. This message should contain all 150 points of data.

Option 2.
Send the data in a format other than as a string. Base64 encoding of the raw numeric values would be ideal, but its going to add extra load to the CPU. Depending on the range of the data (0-359) and the resolution (use 2 DP) you might be able to use just 4 bytes for each data point. Because you are using a fixed length value to represent each point you no longer need separators in you data so a single reading goes from 21 bytes to 12 bytes.

Option 3. Hand off the data to a second processor that is dedicated to sending the data. The Yun would write the data into a shard circular buffer and the other processor (possible another Yun or an ESP8266) would read the data from the buffer, convert it and write it to the server.

I can't say which is best, because I don't know enough about you system, but if you could expand on your details we should be able to narrow it down.


After reading you answer above. If you ranges are 0-10 with two DPs then I would look at option 2 first. If each reading can range from -9.99 to +9.99 you take the absolute value and multiply by 100 and write that as a 2 digit hex number to your message body.

uint8 number = ((reading < 0) ? 0x80 : 0x00);  // Set the high bit for negative.
number += (abs (reading) * 100);
messageBodyString += String (number, HEX);

That will give you 2 bytes per point, and a 300 byte (+overhead) message should be transferable within 5 seconds.


Here is some quick code to explain the encoding and decoding

unsigned int Encode(const float& value)
{
    unsigned int result = (value < 0) ? 0x80000000 : 0x00000000;// Work out and set the sign bit
    result |= (unsigned int)abs(value * 100.0);             // Get the absolute value (ignore sign) and multiply
    return result;
}

float Decode(const unsigned int& value)
{
    const bool isNegative = ((value & 0x80000000)  != 0);       // Get the sign bit
    float result = (float)(value & 0x7FFFFFFF);                 // To remove the sign bit
    result /= 100.0;                                            // Divide to return to the original number of dps.
    if (isNegative)
        result = 0 - result;
    return result;
}
  • sorry for the late reply, was having a hard time absorbing what you're saying since I'm a newbie. I did option 1 already but yun cannot send that much data using the client.get() function from HttpClient library. After deep thinking on your code I understand it already, or somewhat at least. So first is the number should have a high bit (i guess it is also the msb?) to know if it's positive or negative, then I convert the decimal reading to an int, or remove the decimal place and then convert it to a string? I will definitely give this a try thank you so much for the answer! – Ralph Dec 1 '16 at 13:35
  • hello, will i need a separator for this to change it back to decimal numbers? since other hex values are only one character and others are 2 characters. – Ralph Dec 1 '16 at 14:50
  • also another one, how would i check if the hex value reflects the negative number? because when i tested it the same number having opposite signs printed the same hex value, thank you so much for the help – Ralph Dec 1 '16 at 15:10
  • If the most significant bit is set then the number is negative. That's what the 0x80:0x00 is for. You then have 15 bits to hold the value, which is ... oh bugger it isn't enough. You need 2 byte numbers, which give you 4 characters. Sorry. – Code Gorilla Dec 1 '16 at 15:17
  • 1
    hey I just found out that there are some values that doesn't get the right value in encoding, like when the value is -0.53 it returns the result as 52 instead of 53, but it works fine with others, why is this happening? even when I hard code -0.53 it still returns 52 but when I put -0.54 etc it works fine. – Ralph Dec 3 '16 at 16:54
1

Personally for this kind of frequency of data I wouldn't even consider sending it via HTTP. The extra bulk of the HTTP protocol makes the transfer way too slow.

Instead I would set up a UDP-based gateway server which accepts UDP messages from your sensor system and stores the data directly into the database (or wherever you are storing the data) for the website to display it at its leisure.

Yes, it is more complex to create since you need to write the UDP server gateway software. However you then completely remove the massive overheads of a text-based bulky protocol like HTTP.

0

You can use 2d Array for this and send all data at once in 5 Sec. It Read all 3 Axis data each 100Ms and make a matrix of 50 X 3 = 150 Values and then Print it in Serial Port. You can use this in your code to send data to server in desirable format.

int Xaxis=A0; 
int Yaxis=A1;
int Zaxis=A2;
int axisMatrix[50][3]; 
void setup(){
    Serial.begin(9600);
    pinMode(Xaxis,INPUT);
    pinMode(Yaxis,INPUT);
    pinMode(Zaxis,INPUT);}

void loop(){
        for (int i = 0; i <= 50; i++) {
            axisMatrix[i][1]=analogRead(Xaxis);
            axisMatrix[i][2]=analogRead(Yaxis);
            axisMatrix[i][3]=analogRead(Zaxis);
            delay(100);}
        for (int i = 0; i <= 50; i++) {
            Serial.print(axisMatrix[i][1]);
            Serial.print(",");
            Serial.print(axisMatrix[i][2]);
            Serial.print(",");
            Serial.print(axisMatrix[i][3]);
            Serial.print(",");
            Serial.print("\n");
        }} 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.