Yes, there are systems designed to continuously receive such data and the data exchange is standardized, even some Units are standardized.
An example is OPC Data Access for real time values. It seems you want to build a graph, so OPC Historical Data Access is more what you're looking for.
One would set up the data points on the server side once, i.e. configure the type and unit that you're gonna receive on that "channel" or "port" or whatever you call it.
Unfortunately, OPC DA and OPC HDA use XML and building XML is not trivial. It's not very suitable for IoT devices.
Lessons learned: configure the unit on server side and let the client send data only. Do not use XML for data transfer.
Then there is OPC Unified Architecture. They also learned their lesson and removed XML and added security. They now use a binary protocol. The specification is more than 1000 pages, so understanding it will be difficult and is in fact so difficult that even OPC UA implementations of different companies do not work together well.
OPC UA defines 25 data types and ways of defining other data types.
The address space in OPC UA is hierarchical - and probably a device can exist in multiple hierarchies, e.g. physical topology like floors and rooms or logical topology that fits into the structure of the machine.
Lessons learned: make the protocol simple and think about security, i.e. if you want to do URL encoding (
GET), then use HTTPS at least. AFAIK, devices ESP32 support encryption, so it's no longer a problem for tiny devices. People will use very strange data types, like 18 bit integers. Also, do not only provide a flat list of devices, but let them arrange stuff as needed.
MQTT and NodeRed
Now these were quite heavy protocols and models driven by companies. Of course there are also protocols better suited for IoT, like MQTT. MQTT is widely known and uses the also well known Node Red software for handling it. You can define workflows how the data should be handled, converted, stored and whether alarms shall be triggered above or below thresholds.
They already think of connection loss, reconnect and how to send all the data that was already recorded but could not be sent.
Lessons learned: People want more than just data storage and a graphics.
openHAB tries to come up with a List of units. I am a radio amateur. I am measuring electromagnetic fields. I am still missing
- SWR (standing wave ratio)
- dBu (dB µW)
- dBc (dB relative to the strongest signal)
- dBV, dBmV (dB for Volts instead of Watts)
I was working for a company that made software for industry. There are devices that send a measurement value every 100 ms. Let's say this is a simple temperature value of just 1 byte, that's still 315'000'000 records per year.
How do you display such data? On a Full HD screen you would still need to combine 150'000 measurement values on a single column of pixels.
Lessons learned: you need sophisticated data crunching skills to generate the graphics. The users probably want some control over the time span they are viewing. They want zooming and panning.
I was thinking this server can be possibly used by more people to send their data and server displays a chart.
IMHO, sorry, it's hardly needed. People do not want to send their data to a random HTTP server. Some want to set it up at home only, so that no data ever reaches the Internet. Some people want to do more than just displaying a grpahics (especially alarms). MQTT and Node Red runs locally, on a Raspberry, on a server and even in the cloud, whatever you want.
You do not only need
pir=true, pir=1, motion_detect=true, you also need
- an ID for the device,
- a timestamp when the data was recorded
- potentially a way to send bulk data after the server was offline for an hour of maintenance
- a system for units
That's quite an effort. Node Red has > 4000 commits.