OPUS Post processing

toeknee · October 19, 2022, 4:16pm

Has anyone had reliable success with using OPUS to post-process static raw data from the RTK devices?

I’m looking for suggestions from folks that have established a reliable workflow to use OPUS to post-process static data collected with one of the SparkFun RTK devices. (BTW the Facet pair works great in RTK mode!)

I have a couple Facets that I’ve tried to use for static collection, but my success rate with getting OPUS solutions is about 10%. I usually get an abort message from OPUS. I use RTKCONV (demo5_b34g from rtkexplorer) to convert the UBX files to RINEX. I log using the “Logging Default” messages (NMEAx5 + RXMx2)

I’m now butting my head with a couple files that have 4 and 5 hours of collected data, but OPUS rejects them saying there is less than two hours of data.

I don’t seem to have any issues with NR-Canada CSRS-PPP or Australia’s AUSPOS’s service. OPUS has a number of significant advantages for me, as I’m working in the US. It provides output in various US datums, eliminating potential errors I might make in converting datums. Also, the surveyors I work with all use OPUS solutions for our control network, and we want to consistently use OPUS solutions. Solutions from other services are useful as checks, but OPUS is tailored for the US.

I’ve watched the recordings of two NGS-led user forum webinars on using OPUS, and have tried suggestions from there (specifically trimming the first 5-10 minutes out of your log as that early data can be sub-par as the receiver is still settling in.)

Web searches have unearthed blogs suggesting

using the TADJ option in RTKCONV to align the F9P observation times to integer sections (they are often mis-align by a couple microseconds from whole integer seconds) There’s a discussion on the Emlid forums about the Reach and this issue.
converting to different RINEX versions and submitted (2.12 vs. 3.00 vs 3.04).

I’ve tried various permutations of the above suggestions without finding a reliable workflow.

I will say I work in challenging situations. There are trees everywhere where I work. The RTKPLOTs of my data are not pretty. If OPUS aborts on processing my submission because the data is noisy, I accept that. Waiting the leaves to fall off the trees here!

But it usually aborts on with an error message that sounds more like a data integrity or data format issue.

Any thoughts?

sparky · October 19, 2022, 10:15pm

We had issues with a Facet sent to NGS getting mis-aligned data. Paul whipped up an aligner tool that can be found here: https://github.com/sparkfun/SparkFun_u- … main/Utils that may be of some help.

toeknee · October 19, 2022, 10:36pm

@sparky, oh way cool, thanks! I’ll give that a try.

marky9074 · November 30, 2022, 8:11pm

Just wondering how you got on? Also, did you do anything special for AUSPOS as so far I’ve only got CSRS-PPP to work…

toeknee · December 12, 2022, 12:17pm

I’ve been working on other projects and haven’t done much recently. I don’t recall doing anything special for AUSPOS.

marky9074 · January 31, 2023, 7:49pm

Hi @toeknee, when you used CSRS-PPP what was the ‘Fixed Ambiguity’ in the output for static processing? I just realised that most of my datasets return 0% which is a bit disconcerting.

toeknee · February 12, 2023, 2:51am

@marky9074 Latest CRCS I have is from October 3, 2022. So it was a few RTK firmware revs back. 69% fixed ambiguities.

I’m going to go test with f/w 3.1 in a couple days.

Data Start               Data End                      Duration of Observations
2022-10-03 14:57:59.00          2022-10-03 19:27:28.01 4:29:29.010
Processing Time                    Product Type
12:39:46 UTC 2022/10/05 NRCan Rapid
Observations Frequency Mode
Phase and Code Double Static
Elevation Cut-Off Rejected Epochs Fixed Ambiguities Estimation Steps
7.5 degrees             0.00 %               69.19 %              1.00 sec

Sigmas(95%) 0.009 m 0.009 m 0.053 m

toeknee · February 12, 2023, 2:56am

I found another CRCS solution from 9/27/2022 with 70% fixed ambiguities.

Sigmas(95%) 0.010 m 0.008 m 0.059 m

toeknee · February 24, 2023, 11:47am

Good news!

My initial tests with firmware 3.1 are positive, I’ve gotten a decent OPUS solution! The observation location wasn’t the best, had some sky view issues, 81% fixed ambiguities and 63% observations used. I expect to get better when I setup in a location with better sky view.

I now believe that I’d get a great OPUS solution in a location with good sky view when using a Facet with firmware 3.1.

I also post-processed the RINEX file with Carlson’s SurveyGNSS desktop post processing s/w, and the results were within 0.05’ (that’s how US surveyors do things) of the OPUS solution.

I first converted the UBX file with Emlid Studio 1.4 (a wrapper of RTKLIB), checking the “Time Rounding (required for OPUS…)” option. I also opted for only GPS and an interval of 15s, since OPUS currently only uses GPS on 30-second intervals. RINEX 3.03.

I need to work on another pressing project but will test more and report back in a couple weeks or so.

Thanks to everyone at SparkFun who improved the logging in firmware version 3.0 and 3.1. Much appreciated!

Tony

Ingedgar · February 24, 2023, 6:30pm

That’s great News Tony. I apprecciate all your contribution, i bought 2 sparkfun express units because all your positive feedback. Now i’m Wanting a Facet because of the easier Setup, oly thing stopped me from buying was is still now calibrated by the NOAA, hopefully soon. Anyways it seems they are very fine units to work with. Thanks.

Ingedgar · February 27, 2023, 8:24pm

Today i logged 2 hours of raw data and got an Opus solution, with numbers that look good to me. Im going to post process with RINEX data from INEGI base stations, (national geodetic instute from MX) and compare the solutions, im very confident im going to have great results. Later i’m gonna try 6 hours.

I did this with one of ny units, that i have posted have issues with charing but seems is not affecting the performance of the chip, hopefully someone from sparkfun reach to me to solve it.

toeknee · March 12, 2023, 11:48am

Excellent! 97% observations used and 83% ambiguities fixed is good. What were your peak-to-peak errors on the coordinates?

How did your post processing results compare?

I’ve been using OPUS or desktop post processing to check my RTK results on very important points.

I also get multiple RTK readings on important points; sometimes with the base in different locations.

Sometimes I just get bad RTK fixes, and I need to weed the bad fixes out.

For 100s of years, if not 1000s of years, surveyors have been taking multiple redundant measurements to check (and adjust) their work. I’ve learned that GPS is not magic, it’s just another tool, and I still need to take redundant measurements to ensure the quality of the work.

marky9074 · March 20, 2023, 12:37pm

Just to add some improvements from my side with CSRS-PPP, using the below options I managed to get ambiguity better than 80% and the format was usable within Trimble Business Center also.

convbin.exe -od -os -oi -ot -f 2 -ro "-MAX_STD_CP=3 -TADJ=1.0" -v 2.11 FILE.ubx

Eaton_99 · June 18, 2023, 6:34am

I am also trying to get an OPUS solution for a control point, and OPUS keeps aborting. The files upload to OPUS successfully, but then it aborts for various reasons.

I am using a Facet and the firmware version of the u-blox ZED-F9P is 1.32 and the firmware version of the ESP32 is 3.2 4/8/2023.

I am using rtklibexplorer’s modified version of RTKLIB found here:

https://github.com/rtklibexplorer/RTKLIB/releases

And specifically, RTKConv to convert the ubx file to an obs file.

I am also using the aligner tool as suggested from here:

https://github.com/sparkfun/SparkFun_u … ain/Utils

And the new version of the integrity checker: (last update: May 10th, 2023)

https://github.com/sparkfun/SparkFun_u … hecker.py

I have also used the Canadian service CSRS

https://webapp.csrs-scrs.nrcan-rncan.g … LjAuMA…

My understanding of the workflow for setting up a base station or control point where I would set up my Facet set to base mode is:

Set up the Facet directly over the control point with a microSD card in the slot.
Set it to logging defaults (I use WiFI).
Save configuration and exit.
Put Facet into Rover mode and collect data for a period of time. I have tried from 1 to about 7 hours.
Move the resulting UBX file to a computer and convert it to an OBS file using RTKConvert.
Upload this file to OPUS or CSRS and get back a report with the ECEF coordinates corrected for any misalignment of the GPS satellites.

It is this last step where I am having trouble. The OPUS site usually just aborts. And the CSRS site gives me a report which has few if any ambiguities resolved or uses a small subset of the data (maybe this is fine, I do not know). This may be a result of step 2, not setting up the defaults of the Facet correctly (I have performed a factory reset). Or step 5 not processing the UBX file correctly. I have tried setting the interval in RTKConv to various intervals.

So, questions:

How do I get a higher quality survey? Or am I processing the UBX file incorrectly?

The messages I am recording are: NMEA_GGA, NMEA_GSA, NMEA_GST, NMEA_GSV, NMEA_RMC, RXM_RAWX, RXM_SFRBX. Why is NMEA_GSV set to 4 and all the rest set to 1? Should they all be set to 1?

Under the GNSS tab on the Facet the Dynamic model defaults to Portable, should it be set to Stationary?

It has been raining and thunder storming here ( I am in the western US) almost every afternoon so getting a 24-hour or even a 12-hour survey has not happened. I notice that RTKConv can use wild cards for the input file name. So, does it make sense to do several 3- or 4-hour surveys and have RTKConv process them all into one obs file? If I do that, what do I use as the antenna height? I can place the tripod over the same maker quite accurately using a tribrach, but each time the Facet will be a slightly different height. Do I just average the heights of the facet for each survey, and use that average height for OPUS? When I use CSRS how do I input the antenna height and make and model of antenna that I am using?

Any other suggestions would be greatly appreciated.

Thank you.

Eaton_99 · June 19, 2023, 10:32pm

I do not know if this is relevant or not but what does the model symbol mean in this picture?

Thanks,

Ingedgar · June 21, 2023, 9:41pm

Hi, today i post processed a point, and got good results. What i do different from you, is that i use Emlid Studio to Convert from .ubx to Rinex File, with those options selected Later when you have your Rinex, you can exclude some satellites depending in the qaulity of the obervation. I’m in México, in the South west, and even this Far i get great solutions that matches What i got when i post process with INEGI (national Geodetic Institute). Hope it helps.

Eaton_99 · June 21, 2023, 11:19pm

Thanks for your reply. It looks like I should try Emlid Studio.

toeknee · June 24, 2023, 12:59pm

I suggest trying a few things to get an OPUS solution (instead of it aborting).

Upgrade the RTK firmware (that runs on the ESP32). The logging code had been improved significantly in the past couple months.
As Ingedgar suggested, try using the Emlid Studio software to convert the UBX files to RINEX for OPUS. This is what I do and it works best for me. It’s free software; they apparently wrapped an app around the RTKLIB. They apparently give it away for free as it’s mostly open source software. The Emlid devices use the same U-blox chip as the SparkFun devices. Make sure to use the time rounding option to create RINEX for OPUS. I found this to be much more reliable than using RTKCONV and the ubx aligner python script. There’s a way to do the time rounding with RTKCONV too.
I generally wait a couple days before submitting to OPUS. If you submit too early it will abort as it doesn’t have the data it needs yet. There are a few “generations” of improved orbit data that OPUS uses, you can wait hours, days, or weeks and get slightly more accurate solutions. The OPUS site will give you more information.
If it aborts, wait a couple days (again) and resubmit. Sometimes this fixes it.

Other thoughts:

I convert to RINEX (not OBS) so I can examine the RINEX.

I generally try to get 6+ hour occupations. I can toss the first 10 minutes and still have lots of data.

OPUS is GPS only, and sometimes the GPS satellites are just not positioned well when you gather data. There are online services you can use to pre-plan good times.

Sometimes the first few minutes of the log file are bad as the GNSS receiver hasn’t gotten settled in yet. Try tossing out the first 10 minutes of your RINEX file. You could hand edit the RINEX, or, IIRC there’s an option in Emlid Studio to do this when you convert from UBX to RINEX.

Sometimes I find there are gaps in the middle of my data. Perhaps a combination of obstructions (trees?) and satellite orbits cause windows of poor data. These can make OPUS abort. If I examine the data (RTKPlot, Emlid Studio, etc.) and find a 2-3 hour window of good data out of my 6 hour occupation, I’ll toss all the rest of the data and just submit that 2-3 hour window of contiguous good data.

If I have 6 hours of good data, I might submit it three times to OPUS Static (which requires at least 2 hours)

a. the whole 6 hours

b. hours 1-3

c. hours 4-6

and compare all the results.

I might carve out a few 1-hours slices of good data and submit them separate to OPUS-RapidStatic (requires 15 minutes of data) and compare all the solutions. OPUS Rapid-Static uses a different algorithm and it’s a nice check.

About multiple occupations over one mark: I do this frequently, separated by hours, a day, or days. Submit each data file separately to OPUS with the accurate height for that day. Then look at the separate OPUS results for the various occupations and compare. They should be pretty close.

If you have multiple OPUS solutions, you could average the results. OPUS provides covariance matrices in the advanced output option so you can weight your averaging.

I would not combine files from multiple occupations and submit them as one request to OPUS. OPUS knows a lot about the atmosphere, satellite orbits, etc., that differ significantly from one occupation to another. OPUS uses this to refine the solution; I think you’d just mess that all up.

Eaton_99 · June 27, 2023, 12:44am

Toeknee

Thanks for your detailed reply.

I didn’t realize that as of 6/21/2023 the Facet was on firmware version 3.5. I just upgraded. I will see if that helps.

I did download Emlid Studio as Ingedgar suggested and tried it. I think it helps. However, I have been traveling and have not had a chance to thoroughly test it yet. You mention that there is a way to do the time rounding with RTKConv. I have looked and I do not see this option. Is it called something else in RTKConv?

I understand that waiting will give you a better solution. I wish there was a way to test your RINEX file to see if it was even acceptable to OPUS. It is so frustrating to do a 5- or 6-hour survey and wait a couple of days only to find out that OPUS aborts because it is too noisy.

When I use RTKConv to convert my file from UBX to RINEX, RTKConv creates an *.obs file. Isn’t this the file I want? What is the file extension of a RINEX file? How do I get RTKConv to create this type of file?

Thanks for all your help!

toeknee · July 20, 2023, 10:05pm

Eaton99 - take a look at this blog post and the -TADJ option.

https://rtklibexplorer.wordpress.com/20 … in-rtklib/

I pretty just use Emlid Studio to do the conversion anymore.

Before you start collecting the raw data, you can stream the NMEA to any number of apps and look at the S/N and number of satellites to get an idea if your sky view is clear enough.

After you collect the data and before you submit to OPUS, you can use any number of s/w packages (RTKLIB, Emlid Studio, TEQC, along with a number of expensive commercial surveyor-grade s/w packages) to examine and plot out out your RINEX file and look for gaps and noise.

You can also send your RINEX to other processing solutions besides OPUS (Trimble has a free one, Canada and Australia do too.) to learn more about the quality of your data…and maybe get a good-enough coordinate solution for your work if OPUS rejects your data.

Yes, my bad, .obs files are RINEX from RTKCONV. There are a couple RINEX filename standards that I use, and I now expect to see other extensions.

Hope that helps, Tony.