Satelite Azimuth/Elevation jumping around between two different tracks

[vbreyter]

I have observed a very strange behavior with the RTK Express. The azimuth/elevation of some satellite seems to jump back and forth seemingly between two different tracks – see images below. I only observed this after updating the firmware on the device – although I used the device very briefly prior to updating and the problem doesn’t happen with all satellites so i could very well have missed it. Over the course of a 12 hour observation last night (6/5 - 6/6) , I noticed this behavior with B19, B8, B14. During a prior observation I observed this with B14 and R6, R10.

Sometimes after a perdiod of jumping aroudn the satelite starts to follow a single track. Sometimes the “track” it jump to seesm to change to a new track.

I am using the latest firmware (3.3 for RTK and 1.32 for zed-f9p), u-center (22.07) and the TOP106 antenna. Unfortunately I do not remember what the firmware version was before I updated.

[sparky]

This is pretty interesting! I’ve never seen anything like it. You are welcome to downgrade both the ZED and the ESP32 to any previous firmware. You can see all firmwares (for both devices) here: https://github.com/sparkfun/SparkFun_RT … e_binaries

I suspect this has to do with ZED firmware because the RTK Firmware doesn’t (and can’t really) touch the internal tracking algorithms of the ZED-F9x. I would recommend downgrading the ZED to a few other earlier versions to see if the behavior changes. If you continue to see oddities, we can help you get in touch with a u-blox FAE.

[sparky]

Just to add a few more questions to the list: this isn’t some weird multipath issue is it? Antenna issue? I don’t think so, but (just to cover the bases) do you have a clear view of the sky? Might you have another L1/L2 antenna that you could swap out to eliminate the antenna as the problem?

[vbreyter]

I downgraded to 1.31 for RTK and 1.30 for ZED and will leave it running overnight and report back.

As you your other questions.

1. The antenna is setup with an almost completely unobstructed view of the sky. The only obstruction is a house ~30 ft directly north . Given I’m located in the northern US (Northeast), I figured there are no GNSS satellites in the immediate northerly sky anyway. I can try setting up the antenna in a different location tomorrow that is further away from the house.

2. I don’t have another antenna but I can setup a Facet tomorrow in the same location (after I move the current TOP106) and see if it shows the same anomaly.

[vbreyter]

I downgraded the Express unit to 3.1 RTK firmware and 1.2 for ZED. I saw the same anomaly described in the original post. I then ran both an Express connected to a TOP106 and a Facet for about 12 hours. Both were freshly flashed with v3.3 RTK and 1.32 ZED firmware. The Express/TOP was located in same location that was generated thy Skyplot in the original post; this location has unobstructed views of the sky except directly to the north, where there is a ~25ft house ~30ft directly north. The Facet was placed bour ~50 feet south of the Express/TOP; this location is similar with wide open view of the sky with the nearest obstruction being a the same ~25ft house located ~80ft directly to the north. As you can see below, both sky plots show the anomaly, however it’s interesting that the Facet sky plot does seem to be missing one instance of it. This instance was very early on in the run and the Express started logging a few minutes earlier so perhaps the Facet simply wasn’t on to receive whatever signal is causing this.

[PaulZC]

Hi,

This is fascinating stuff.

Can I ask how you are collecting the satellite sky position data? Are you connecting u-center to the UBLOX CONFIG port and doing all of the logging and visualization directly in u-center? Or are you logging NMEA and/or UBX-NAV-SAT to microSD card and then replaying the logged data through u-center?

I think this has to be a multi-path reflection phenomenon. Maybe the house wall to the North has reflective foil in it? Or is metal-clad? Or otherwise creating a strong reflected signal?

The only other explanations I can think of are that the GNSS module is somehow getting the satellite identifiers confused, or mis-decoding the satellite ID. Or that the satellites themselves are broadcasting the wrong identifier? But both of those would mean something had gone seriously wrong.

It depends on the satellite orbits, but you will have satellites to the North of you. I think the ‘hole’ in your plots is caused by the house blocking the signal.

Best wishes,

Paul

[vbreyter]

PaulZC:
Can I ask how you are collecting the satellite sky position data? Are you connecting u-center to the UBLOX CONFIG port and doing all of the logging and visualization directly in u-center? Or are you logging NMEA and/or UBX-NAV-SAT to microSD card and then replaying the logged data through u-center?

I've tried it both logging through the ublox usb port, and through the micro SD. Same issue.

PaulZC:
I think this has to be a multi-path reflection phenomenon. Maybe the house wall to the North has reflective foil in it? Or is metal-clad? Or otherwise creating a strong reflected signal?

The house in question is my own so I can, with confidence, say that it's not metal clad nor does it have any metallic/foil wrap. Regardless though, I don't think this can be explained by a muti-path issue (at least not directly, see next paragraph). The u-blox receiver / TOP antenna is not actually "tracking" the satellites. The antenna/receiver picks up the signal from the sat but it doesn't know which direction the signal came from. For that you would need an antenna array and a phased array capable receiver. So, even if there is a MP issue due to a reflection off the house, the receiver would just see that as a signal that's taking slightly longer to get to it and this would introduce some position error into the fix. The receiver only knows where the signal/satellite is from the ephemeris data it receives.

PaulZC:
The only other explanations I can think of are that the GNSS module is somehow getting the satellite identifiers confused, or mis-decoding the satellite ID. Or that the satellites themselves are broadcasting the wrong identifier? But both of those would mean something had gone seriously wrong.

I'm totally guessing, but it *seems* like the receiver is somehow getting confused on the timestamp of when it received the signal from the sat in question and momentarily puts the sat location to a different part of the orbit (or different day altogether). Either that or it's getting confused which sat the signal is coming from and puts the sat on a totally different orbit -- although not sure why it would have the connecting lines then. As you said though, I don't think this is caused by the sat sending incorrect data as that would be a *serious* problem that would have been caught. My guess is that there is some bug in the u-blox firmware. Given that I get the same result with older firmware, if it is a bug, it has been around for a while and something about my specific config/setup is causing it to manifest. So perhaps there is some MP issue that's causing the receiver to run a correction/compensation algorithm and that algo has a bug in it. Again totally a guess though.

PaulZC:
It depends on the satellite orbits, but you will have satellites to the North of you. I think the ‘hole’ in your plots is caused by the house blocking the signal.

I also don’t believe that there would be any satellites directly to the north of me. The hole on the sky plot is due to the fact that GNSS sat orbits are inclined around 55-65 degrees, and given my latitude, that means they should just make it overhead , which is what you see in the plot. If you took the same plot at the equator you wouldn’t see any sats at the very top and bottom of the plot and as you move away from the equator the hole moves down from the very top/bottom of the plot, until it’s directly overhead when you get to the north/south pole. You can run the predictive sat visibility analysis in rtklib to verify.

Edited for clarity/quotes.

[vbreyter]

sparky:
Just to add a few more questions to the list: this isn’t some weird multipath issue is it? Antenna issue? I don’t think so, but (just to cover the bases) do you have a clear view of the sky? Might you have another L1/L2 antenna that you could swap out to eliminate the antenna as the problem?

Sparky, given that I’m getting the same issue with two different devices (the Express/TOP combo and the Facet) and from a second location, any other ideas to try to resolve this.

[TS-John]

Hey vbreyter.

Sparky is out until the 20th. If you could stand by until then that would be the best for the time being.

I have your case open in our CRM as well and will be reaching out there.

Thank you for your patience and understand here.

[toeknee]

You could try isolating and manually examining the NMEA $G*GSV messages for just one of the space vehicles that appears to be jumping in the skyplot. Do the az and el number for that SV jump around in the NMEA messages? You could even write a little code, create a CSV file for that one SV, and plot it in excel or something. This would rule out a problem with the Skyplot plotting too.

[PaulZC]

That’s an excellent idea - thanks Tony!

@ vbreyter : can you please Private Message me some of your jumping track data, so I can take a look? Thanks!

Best,

Paul

[toeknee]

There are at least four different numbering schemes for the SVs (space vehicles, aka vehicles, birds, satellites). Remembering coding mistakes I and my colleagues have made in the past, it’s possible that there is just confusion somewhere in the SV numbering. I wouldn’t expect a bug in the F9P firmware; I’d guess it’s well-tested mission critical code. U-center, however, is more of a debugging & development tool. Ditto for RTKLIB and other “user software” that creates the sky plots. The plotting errors look too regular to me to be random message corruption. But anything is possible!

Most of the numbering schemes we deal with are virtual, that is they refer to a functional SV occupying a slot in an orbit. Different physical SVs get moved in and out of the functional orbital slots as newer SVs come on line and older SVs go off line. Users, like us, almost always just need to know the functional, virtual, SV numbers.

- the orbit slot numbers assigned by the operator of each constellation. This is a virtual, functional identifier. IIRC these are also referred to as the PRN numbers.

- NMEA renumbering of the orbital slots (eg. GLOSNASS slot 1 is NMEA slot 65).

- UBX identification of the slots in the UBX-NAV-SAT messages.

- the unique 'serial number' of each physical SV as assigned by the constellation operators, sometime before or around launch time. These uniquely identify the physical satellite Users like us rarely use these numbers; it's probably only interesting to us as newer, more capable SVs (eg. with L5, L1C) replace older SVs in the orbital slots.

The UBX F9P Interface Description document (I have v1.32) relevant sections are

1.5.2 GNSS identifiers

2.7.15 GSV

3.15.16 UBX-NAV-SAT

Here is just one of many web pages about the SV numbering.

https://continuouswave.com/ubb/Forum6/HTML/003694.html

[vbreyter]

Excellent suggestion toeknee!

Through trial and error I managed to put together the following awk script to print out any instance where the el or az changes a lot between subsequent gsv messages. I filtered out anything below 10 el to get rid of instances where the sv is just coming into view and also above 80 because a) az changes quickly at those els and b) looking at my slyplot the situations I was investigating were all below 80 el . The result only split out a few instance where the az crossed 360 but nothing showing the kind of jumping around that I’m seeing on the skyplot. Are there any other messages that I should be looking at to see if there is something else that’s causing the jumps?

function abs(v) {return v<0 ? -v : v}
function calc_trigger(el) { if (el >=80) {return 8}else if (el >=70) {return 6}else if (el >= 60) {return 4}else{return 3} }
BEGIN { FS = ","; }
/^\$G.GGA/ { time = $2 }

/^\$GPGSV/ { #for GPS
        for(i=5; i<=(NF-3); i+=4) { #look at each set of svid,el,za,co values in a GSV line
                if (g_el[$i] && abs(g_el[$i]-$(i+1))>3){ #test if elevation has a large delta from previous value for SV
                        printf "%s: G%s: EL %s -> %s\n ", time, $i, g_el[$i], $(i+1)
                }
                g_el[$i] = $(i+1) # update el value for SV

                trigger = calc_trigger($(i+1)) #calculate threshold based on elevation

                #test if az has a large delta from prior value for SV, filter anything below 10 el or above 80 el
                if (g_az[$i] && (abs(g_az[$i]-$(i+2)) % 360) >trigger && ($(i+1) > 10) && ($(i+1) < 80)){
                        printf "%s: G%s: EL: %s, AZ %s -> %s \t %s \n ", time, $i, $(i+1), g_az[$i], $(i+2), g_az[$i] - $(i+2)
                }
                g_az[$i] = $(i+2) #udpate az for SV
        }
};

#repeat the above section for the other constelations