Openlog Send Log File WiFi

Other Miscellaneous
1

Setting up a seismic sensor with ESP8266 (NodeMCU 1.0), MPU6050 and Openlog. Sensor gets roughly 10 readings per second and writes it to openlog. Lines of the log file is like:

 21:38:50,0.08,0.00
 21:38:50,0.06,0.00
 21:38:51,0.04,0.00
 21:38:51,0.01,0.00

You can imagine, the log file gets some hundreds MB with millions of characters. I am planning the ESP8266 to send this log file every hour or every n hours to some network drive, and delete and restart the log file on openlog SD. If i give up openlog and send the data directly to network drive, than the network traffic is very high.

The large file sample on github is 1000 characters long. I think i can handle the buffer size with:

read File + START+ LENGTH TYPE

sequential read of file in chunks. But i really don’t know the buffer size of Serial, what should be the size of the read chunks? Or is there any other better way?

Thanks…

2

Why store/send raw data, when 99.99% of it shows nothing but noise?

With a little added intelligence, the system could send “event records” instead.

3

What a smart idea… When you are too much into a project, you can get lost even into the simplest solution. I will go into that direction. Thanks @jremington.

But the problem still stands for some other projects that might need this; for future problem owners.