I really need to read a file containing 3 to 5,000 bytes. I don’t want to create a buffer that large. The way I read Example 5, the largest buffer I could create (if I had the memory is 255 bytes. I don’t need all the data in a buffer at once. I could read 40 to 60 bytes at a time into a buffer and send the buffer contents via radio to a base station. Repeating the process, starting at the next character until the file is sent.
The largest number of characters I’ve read using the code in Example 5 is 255. I created these files using the approach in Example 2 as a guide and the files can be read fine on a laptop.
Thoughts?
Qwiic is I2c protocol. You can send as much data as you like in blocks.
Search on internet on “large data over I2C” and there are many articles with sample code how they do that.
Thanks for the quick reply! The library (Sparkfun Qwiic OpenLog) as written handles chunking up files as large as 255 bytes into 31 byte packets and sending them. My problem with the library seems to be there is no persistent way to record the last byte read from a larger file and returning to read the next buffer full from the next position. I could even do the book keeping to figure out where to move the file index to begin each buffer full, but there is no library function to set the first byte to read in a large file. For instance, you would read the first 80 characters of a large file using myLog.read(myBuffer, myBufferSize, myFile); Once read, you would print or send those characters, clear the buffer and get the next buffer fill with a command like myLog.read(myBuffer, myBufferSize, myFile,80); There is even a commented out function call that looks like that in the example with a note that says “Doesn’t work yet”. It has been there for a while, so I wonder if this product is orphaned or EOL’ed.
Understand your point. I think this Openlog was created with “storing large amount of data” in mind and less with reading it back.
Looking at the firmware the start position (INT) can be set in the Openlog firmware (Qwiic_OpenLog/Firmware/Qwiic_OpenLog at master · sparkfun/Qwiic_OpenLog · GitHub), but it is not implemented in the Sparkfun_qwiic_openlog_arduino_library.
Thanks for the pointer to the firmware. Hacking the firmware wasn’t on my agenda, but if I can’t get another solution… Seems like they want to store large amounts of information and then remove the MicroSD card to read it on a laptop, or for persistent, small configuration files. I think they are forgetting a pretty substantial use case. Extended data collection in locations that are difficult to access directly and continuously. Think of atmospheric data in the the canopy of a remote forest. You don’t want to climb an 80 foot Doug Fir very often to collect the MicroSD and you don’t want to put another computer near the ground to collect the data into. Just go back once a month, download via XBee or other radio and change to the new file for next months data. When you finally collect the sensor package you have the MicroSD as backup, but you can analyzed the downloaded data in the meantime. Might be useful in a lot of applications.