Postcard: Rinex SD card output

Hi guys,

I have the RTK Postcard and Portability Shield combo. The NMEA output is logged to the SD card out of the box, but I want Rinex files instead. I want to use it for PPK in RTKLIB.

The RTK Everywhere docs tells me to use u-blox software to change it, but I’m assuming that doesn’t work for Quectel chips. I haven’t found anything equivalent in QGNSS.

Does anyone know how to log Rinex files to the SD card instead of NMEA output?

Instead of logging NMEA, you can configure the module to output RAWX and SFRBX messages.

That’s incorrect for the LG290P used in the PostCard.
The PostCard can save RTCM which you can later convert to RINEX format for PPP post-processing analysis.

Take a look at the 7th post here Postcard firmware update and assembly - #7 by sparky which has some notes (namely use base mode and RTK2go)

https://software.rtcm-ntrip.org/browser/ntrip/trunk/rtcm3torinex/lib/rtcm3torinex.c

1 Like

Sorry, I think my first post might have been badly worded. I want to use my Postcard as a rover, logging RINEX (or smiliar) files to the SD card, then combining my rover RINEX file with observation files from a comercial 3rd party base station in RTKPOST.

Using the serial terminal, I enabled the following RTCM messages in the RTCM rover menu:

I’m still logging NMEA messages like before:

The Postcard is still logging to a txt file on the SD card, but there seems to be what looks like binary or some sort of different encoded text on some lines. After turning on RTCM messages, I also have troubles with the Postcard crashing and bootlooping, requiring me to power cycle it to make it work again.

The txt files cannot be read by RTKCONV and not by RTKPOST.

Screenshot of the txt file:

Trying to convert the txt file to RINEX using convbin (see docs here):

> .\convbin.exe -r rtcm3 ../SFE_Postcard_250207_071006.txt
input file  : ../SFE_Postcard_250207_071006.txt (RTCM 3)
->rinex obs : ../SFE_Postcard_250207_071006.obs
->rinex nav : ../SFE_Postcard_250207_071006.nav
->sbas log  : ../SFE_Postcard_250207_071006.sbs

rtcm open error: SFE_Postcard_250207_071006.txt


> .\convbin.exe -r rtcm3 -o myRinexFile.rnx ../SFE_Postcard_250207_071006.txt
input file  : ../SFE_Postcard_250207_071006.txt (RTCM 3)
->rinex obs : myRinexFile.rnx

rtcm open error: SFE_Postcard_250207_071006.txt

Anyone have any recommendations for how I can do PPK on the output of the Postcard + Portability Shield in rover mode?

IMHO, You are asking the PostCard to do too much in “real-time”

As a reference, below are the NMEA Settings that I use for the PostCard as a RTK Rover:

1) Message RMC: 1
2) Message GGA: 1
9) Message GST: 1

I would prefer to have GSV every 10 seconds, but the LG290P Firmware only allows 1/Off at the moment, so GSV stays OFF for now.

My opinion is: If my data collector doesn’t need the NMEA message - then don’t send it.
This is especially true for you since you want to also save the RAW Observables for post processing.

The Screenshot of your text file :
The GNSS Observables are not going to be in a “Human Readable” format.
I don’t think you want the NMEA saved however.

Post Processing from Postcard:

As @swells stated in this post, the conversion is successful from a PostCard that was logging RTCM in Base Mode. So I expect you can be successful with your PPK goals, once that specific workflow is established.

My assumption is: You want to post-process GNSS data with PPK for PostCard positions that were originally computed in real time using RTK.

Thanks @rftop!

I’ll try to fully disable NMEA messages later today to see if that helps.

Did the selection of RTCM messages in the screenshot seem reasonable?

Would changing the frequency make any difference to the results? I see the documentation says that the frequency is 1Hz in base mode because it doesn’t improve the base station to increase the frequency, but is that the same for rover mode? Mine is currently set to the default value of 2Hz.

I want to create a portable location logger that I will have on my person (in pockets, backpacks etc). I know that’s probably not what the Postcard was made for, and that’s totally fine, but I’ll try to make it work - it’s a fun project! It’s actually looking doable! I’m getting DGPS or RTK floats in my jacket pocket and RTK fixes in my backpack.

The optimal solution would be something that can give me a good location in real time using RTK, and that I could do post-processing on back home to make the result better and to fill in the gaps where RTK for some reason didn’t work or was unavailable. My dream solution would be a self-contained device that logs both GPX/KML and RINEX/RTCM/raw files at the same time, and with a screen to show me accuracy, speed and elevation. It would also be useful to save BSSIDs of WiFi access points around me to use for geolocation inside buildings (using Google Geolocation API or similar). That’s what I’m working towards. I’ll try to get as far as possible with the standard firmware, but I’ll probably make some changes to the firmware to fit my needs.

1 Like

I’m still getting NMEA messages in the SD card txt file, even though all NMEA messages are disabled. Any idea what I’m doing wrong?

It’s less than before, but there still are NMEA messages mixed with the other binary data. convbin is still unable to read the file.

NMEA messages at the top/start of the txt file:

NMEA messages in the middle of other binary data:

This might be a quick check:
Put the PostCard in Base Mode, and perform a quick survey.
Check if your software workflow will convert the Base files to RINEX.

2 Likes

In base cast I am seeing same thing it records to a txt file and I’m on latest release client

1 Like

I was able to take the txt binary and put into rtkconv, selecting rtcm 3 as type and generate rinex files. this was a 26 minute observation. I then took it into Emlid studio aligned it to even second, and made it 15 second intervals. Tomorrow I will throw it at OPUS and see what happens. I was using default settings in base cast mode.

1 Like

The reason for convbin not working was human error: I was using a forward slash instead of a backward slash in the input file path. After correcting that, convbin can read my rover txt files. I also tested in base mode as per @rftop’s suggestion, which also works in convbin.

Both modes still give NMEA sentences in the output even though they’re turned off, but convbin is able to ignore the NMEA messages.

I’ll consider the original issue of getting RINEX output resolved, but there are still some unresolved questions at the bottom of this post. I’m also getting terrible results from this process, also described at the bottom of this post. Here’s the tutorial for RINEX output:

  1. Enable RTCM3 messages on the Postcard using the serial terminal. Suggested messages:

:x: 1005
:x: 1006
:white_check_mark: 1019
:white_check_mark: 1020
:x: 1033
:x: 1041
:white_check_mark: 1042
:x: 1044
:white_check_mark: 1046
:white_check_mark: 107X
:white_check_mark: 108X
:white_check_mark: 109X
:x: 111X
:white_check_mark: 112X
:x: 113X

  1. Disable as many NMEA messages as possible, because the Postcard will crash and bootloop if it’s trying to write too much to the SD card at the same time. Trying to avoid crashes by reducing the output is hard, because of different parts using different NMEA sentences. Suddenly the Portability Shield stops showing number of sattelites, and the SD card output file shows its date as in May 2027, then SW Maps stops showing data and so on. I’m still trying to figure out the best combination of NMEA messages here.

  2. After logging your travels, extract the txt file from the SD card and run the following command: .\convbin.exe -r rtcm3 SFE_Postcard_250207_071006.txt . Convbin is available here: Releases · rtklibexplorer/RTKLIB · GitHub

  3. The above command will output observation and navigation files that can be used in RTKPOST (also available from the link above).

Remaining unanswered problems:

a. How can NMEA output be fully turned off?

b. Is there any way to log a lot without crashing the Postcard? It would be great if I was able to log everything at a high rate, but that might realisticly not be possible. Here’s an interesting commit for “setMessageRateOnPort”, maybe that can be used in the next firmware release: Add setMessageRateOnPort. Requires LG290p firmware v4 · sparkfun/SparkFun_LG290P_GNSS_Arduino_Library@4557b39 · GitHub

c. What is the best logging frequency for RTCM data?

I’m also getting truly terrible output from this process.This is a 4km trip, and the positions are very chunked for some reason. According to itself (the Portability Shield screen and SW Maps), the rover had acceptable accuracy the entire trip. This might be my own fault, I’m very new to the world of advanced GNSS devices. Maybe some of you recognize what’s wrong?

The results should have been something like this:

Hi folks,

I love where this conversation is headed, please keep it coming. As we veer into firmware bugs, please consider reporting single, reproducible issues and we’ll get them rolled into the release candidate. We’re currently focused on cellular and network layer issues but we can tackle Postcard specific issues as they arise.

How can NMEA output be fully turned off?

On the LG290P, this is problematic since it does not have a binary sentence that reports satellites in view. We have to calculate that from the GSV sentence. So if we “truly” disable NMEA, a fair amount of user interface breaks. But if you can make the case for a feature, we’ll break it.

Postcard will crash and bootloop if it’s trying to write too much to the SD card at the same time.

This is not normal. Please create an issue and help us replicate it.

1 Like

Hi @sparky,

Just clarifying some minor things to avoid misunderstandings:

  1. I’m now trying a different and better SD card formatted with the official “SD Association” formatting tool just to rule out other causes for the bootloop. I’ll report the issue to GitHub if it continues.

  2. I’m able to turn off GSV, but there are a few others that don’t turn off. One of the proprietary NMEA messages that won’t turn off is PQTMPVT, which contains the number of satellites in use. That might be able to replace GSVs number of satellites in view ? PQTMPVT also contains date and time. The SD card file name doesn’t give the correct date if I disable RMC. In other words, PQTMPVT (which is already forced on) can replace the need for GSV and RMC. Just an idea.

  3. For me it’s not necessarily about disabling the NMEA output internally, but disabling the writing of NMEA to the SD card. Even better would be to write NMEA to one file and RTCM to another file. They use different encodings/types but are written to the same file. That creates problems in many file readers. Example trying to read NMEA from a file that also contains binary RTCM in Python:

f = open("input.txt", "r")
Output: UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1850: character maps to <undefined>

f = open("input.txt", "r", encoding="utf8")
Output: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd3 in position 1821: invalid continuation byte

f = open("input.txt", "r", encoding="utf8", errors="ignore")
Output: $GNTXT,01,01,00,ESP_RST_POWERON*1C

Is there a future possibility of separating NMEA and RTCM output to two different files written in parallel?

1 Like

One of the proprietary NMEA messages that won’t turn off is PQTMPVT, which contains the number of satellites in use

Sorry, I said binary when I should have said proprietary. You are correct. SIU and SIV. To get SIV, we have to parse the GSV sentences. Turning off PQTMPVT leads to other issues, but talking with another engineer here, I think there’s a good way to filter this message out of the circular buffer that gets sent to the myriad of potential consumers (logging, Bluetooth, TCP, WiFi, etc). See this issue to track it.

Is there a future possibility of separating NMEA and RTCM output to two different files written in parallel?

It’s possible but unlikely. We have to draw the line between being a GIS device and a GIS data collector. It’s better to rely on the vast set of post processing tools available to the GIS industry rather than try to build those tools into the RTK Everywhere firmware. Said differently, the tools working on your logs should have parsers built into them to gracefully handle what is thrown at them.

3 Likes

So true and I love to hear that :slight_smile:
Scope Creep can take a great product and turn it into a decent product that does more random stuff.

Admittedly, I’m the guy who will probably be asking for a new feature tomorrow :joy:

2 Likes

I fully understand where you’re coming from, but I want to present a different perspective:

The current TXT file written to the SD card combines UTF-8 text and binary data in a way that makes parsing difficult. Because the UTF-8 and binary data are interleaved, there is a chance of data loss during reading. In my experience, I often lose the first NMEA sentence after every binary data segment. I also have troubles with binary data in the middle of my NMEA sentences.

The Postcard with the Portability Shield and RTK Everywhere serves as a data source. While post-processing tools can handle many things, a clean data separation at the source would improve usability and reduce parsing issues downstream.

Example of binary data cutting off an incomplete GSV sentence. This is really hard to work around, and I think the only way is to fix this in the source (meaning the firmware).

$GBGSV,**removed for screenshot**
$GBGSV,**removed for screenshot**
$GBGSV,**removed for screenshot**
$GBGSV,3,2,11,28,18,07Ë~@¸õ\Ó@.......
$GNGGA,**removed for screenshot**
$GNRMC,**removed for screenshot**

Would love to hear your thoughts on this - perhaps there’s a way to implement this efficiently without overcomplicating the firmware.

With that being said, I have created a small code snippet to reliably separate the NMEA and RTCM3 output from the SD card, with the exception where RTCM3 and NMEA are fully mixed and cuts each other off. If anyone else is experiencing data loss from reading the txt file from the Postcard, check it out here: GitHub - mamk95/SplitNmeaAndRtcm: Takes an input file containing mixed and interleaved NMEA (UTF-8) and RTCM3 (binary) content and separates it into a .nmea file and a .rtcm3 file.

my first attempt at 26 minute rapid static on opus failed. So i tweaked the messages and will try again.

Do you know why it failed?

I’m really hoping that @sparky and Sparkfun can fix the separation of RTCM and NMEA in the firmware (the more I look at it the more I would call this a bug fix rather than a feature request), but in the meantime you can try the software I wrote to combat this: GitHub - mamk95/SplitNmeaAndRtcm: Takes an input file containing mixed and interleaved NMEA (UTF-8) and RTCM3 (binary) content and separates it into a .nmea file and a .rtcm3 file. You can easily use the executable directly (see the folder “executable”, just download it and drag and drop your txt file onto the exe file), alternatively download .NET 9 SDK and run it using dotnet run input.txt.

I’ll try to make some small adjustments to the software this weekend. Currently there will be a bit of data loss where e.g. a RTCM3 line cuts off a NMEA sentence, but I might be able to work around it in code in a few days. It seems that the output from my software is a correct RTCM3 (although missing a few messages), and mostly correct NMEA sentences with a few corrupt messages.