0

I want to get some JSON data from a HTTPS secured Webserver using a ESP8266. For some reason I'm getting an empty response, I believe there's an error parsing the GET request. In Chrome I get the JSON response without problems.

GET Request:

httpsClient.print(String("GET ") + path + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +               
               "Connection: close\r\n\r\n");

Host and Path:

const char *host = "www.adler-mannheim.de";
const char *path = "/jsonapi/game/current";

UPDATE:

I tried fetching some JSON from another Server and it worked finde, so i can rule out my code. Fiddling around with the host and path I observe this:

  • Hostname without "www" results in a 301

  • Hostname with "https://www" results in no HTTPS connection at all

So it seems that a "www" is required although it unfortunately leads into a empty response.

The output is as follows:

headers received
reply was:
==========
==========
closing connection

In the first case, without using "www" in ahead of host there's some 301 HTML in between the two lines of "=".

Whole function:

void connect() {
  WiFiClientSecure httpsClient;    //Declare object of class WiFiClient

  Serial.println(host);
  httpsClient.setInsecure();
  
  Serial.print("HTTPS Connecting");
  int r=0; //retry counter
  while((!httpsClient.connect(host, httpsPort)) && (r < 30)){
      delay(100);
      Serial.print(".");
      r++;
  }
  if(r==30) {
    Serial.println("Connection failed");
  }
  else {
    Serial.println("Connected to web");
  }
  
  String getData, Link;

  //GET Data

  Serial.print("requesting URL: ");
  Serial.println(host+Link);

  httpsClient.print(String("GET ") + path + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +               
               "Connection: close\r\n\r\n");

  Serial.println("request sent: ");
  Serial.println(String("GET ") + path + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +               
               "Connection: close\r\n\r\n");
  while (httpsClient.connected()) {
    String line = httpsClient.readStringUntil('\n');
    if (line == "\r") {
      Serial.println("headers received");
      break;
    }
  }

  Serial.println("reply was:");
  Serial.println("==========");
  String line;
  while(httpsClient.available()){        
    line = httpsClient.readStringUntil('\n');  //Read Line by Line
    Serial.println(line); //Print response
  }
  Serial.println("==========");
  Serial.println("closing connection");
    
  delay(2000);  //GET Data at every 2 seconds
}

UPDATE 2:

I tried the following code from the ESP8266HTTPClient Library examples:

void connect2() {

  WiFiClientSecure client;

  //client.setFingerprint(fingerprint);
  client.setInsecure();
    
  HTTPClient https;

  Serial.print("[HTTPS] begin...\n");
  if (https.begin(client, "https://github.com/esp8266/Arduino/issues/3417")) {  // HTTPS

    Serial.print("[HTTPS] GET...\n");
    // start connection and send HTTP header
    int httpCode = https.GET();

    // httpCode will be negative on error
    if (httpCode > 0) {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTPS] GET... code: %d\n", httpCode);

      // file found at server
      if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
        String payload = https.getString();
        Serial.println(payload);
      }
    } else {
      Serial.printf("[HTTPS] GET... failed, error: %s\n", https.errorToString(httpCode).c_str());
    }

    https.end();
  } else {
    Serial.printf("[HTTPS] Unable to connect\n");
  }

  Serial.println("Wait 10s before next round...");
  delay(10000);
}

This one gives me a 200 but no payload. Other sites work

UPDATE 3:

I had a look on the headers.

https.headers()

This gives me 0 headers on every page, even when there's a payload.

I also had a closer look at the payload that returns from my desired api and its filled with NULL. In a weird way this is only the case with my desired page and only my desired subpages. Requesting something more at top level e.g "https://www.adler-mannheim.de/jsonapi" which is basically a SilverStripe Welcome page, I get a fully fetched beautiful HTML payload.

I got literally no Idea why. Do I have to set a specific User Agent or something so that the Server responds?

10
  • I assume it's an empty response since it says Reply (nothing in between), whereas from other sites theres a response in between. I'll edit the question to make it clear – Christopher Apr 19 at 9:26
  • https:// is not part of the hostname. It's the protocol indicator for a URI. – Majenko Apr 19 at 9:49
  • The 301 response will also have a "Location:" header which tells you the real URI you should be accessing. 301 is permanently moved, so it is safe for you to replace your URI in your code with the one provided in the Location header. – Majenko Apr 19 at 9:53
  • The 301 points to the exact URL which I'm trying to access (adler-mannheim.de/jsonapi/game/current). It also works fine in a Browser. – Christopher Apr 19 at 10:08
  • I guess you don't wait for the response long enough – Juraj Apr 19 at 10:10
1

Okay I have no Idea what I changed apart from deleting some imports which might have collided, but now it works. Regular call with "https://www" does the thing.

Thanks anyways

2
  • It would be better to have a more complete answer, assuming it can be of use to someone else. Otherwise the question and answer both are probably better off deleted. – timemage Apr 19 at 15:17
  • the version with WiFiClient or the version with HttpClient? – Juraj Apr 19 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.