RTK Reference Station Reboots

ODMerdin · January 15, 2025, 7:53am

Hello everyone,

I am currently commissioning our second RTK Reference Station but have encountered a troubling issue: whenever I enter Survey mode, the station reboots continuously and never completes the process. Originally, the station shipped with firmware 3.5, and I tried upgrading it via the Wi-Fi config page, but it would reboot as soon as I clicked “Install.” As a workaround, I updated the u-blox ZED-F9P module firmware to L1/L2 HPG 1.51 using u-center over USB and then upgraded the Reference Station firmware to 4.1 with RTKUploader over USB. Following these upgrades, the firmware versions displayed correctly on the Ethernet config’s web interface, but after leaving the station off overnight, the Ethernet config page no longer shows the u-blox firmware. Additionally, the Wi-Fi config page still fails to work.

Now, whenever I try to cast to RTK2go and switch to Survey mode, the station reboots continuously, never staying on long enough to complete the survey or establish a data stream.

Has anyone experienced something similar, or does anyone have advice on how to fix this? Any help would be greatly appreciated. Thank you in advance for your support.

ODMerdin · January 15, 2025, 9:04am

Hello everyone,

After a thorough investigation, I have observed the following issues.

Whenever the Ethernet cable is connected, I cannot access the RTK Reference Station through the Wi-Fi config page.
If I do not enable the NTRIP server option, the system functions normally.
Once I enable the NTRIP server option, the station briefly starts transmitting after a survey and displays a message about obtaining an IP address before abruptly rebooting.

I am currently suspecting that incorrect authentication credentials might be causing this repeated reboot cycle. Has anyone else experienced similar issues, or does anyone have suggestions on how to address this problem? Thank you in advance for your help.

PaulZC · January 15, 2025, 10:31am

Hi Oğuz ( @ODMerdin ),

This could be a RAM issue, but I am not sure. Please follow the link below for more details. Please try disabling Bluetooth and let us know if this corrects the issue.

I hope this helps,
Paul

Sparkfun Reference station wifi set up GPS/GNSS

Hi Praveen, Thank you. I can now see the same issue: [image] Previously, I did not have an SD card inserted. I see the error now, after inserting an SD card. I think the ESP32 is running out of RAM. The SD task is using some extra RAM and this causes the WiFi task to fail. If I disable Bluetooth to save RAM, the unit is able to connect to WiFi and I can use PuTTY to view the NMEA data. Please try replicating this to confirm this is the issue: Disable Bluetooth: s b b You should see “b) Set Bluetooth Mode: Off” in the System Menu Exit the menu (this saves the settings): x x Then force a system reset with: s d r RAM use on the Reference Station is on a “knife edge”. We know there is not enough RAM to run ESP-Now and Bluetooth simultaneously. I think in your case, enabling TCP Server is using additional RAM and preventing WiFi from working… Thank you, Paul

ODMerdin · January 15, 2025, 12:09pm

Hey Paul,

Thanks for your reply. Does disabling Bluetooth on the Wi-Fi config page have the same effect as turning it off through the terminal? We tried turning it off in the Wi-Fi config, and for a brief moment, the station started casting before it rebooted. If you think I should try disabling Bluetooth through the terminal, I can give that a shot.

Right now, if we run the station without an Ethernet cable, there is no rebooting, but when we plug in the Ethernet cable, it reboots just as it’s supposed to start casting.

Thanks again,
ODM

PaulZC · January 15, 2025, 2:08pm

Hi ODM,

I think I am seeing the same thing. And I believe it is RAM related.

My Reference Station is running v4.1. The ZED is running 1.32, but I don’t think that will make a difference.

I have an SD card inserted.

I have Ethernet connected.

I connect to the CONFIG ESP32 USB port with Tera Term at 115200 baud.

I reset the settings to the default (s r y).

The unit reboots and Base mode starts. I let the survey-in complete. The OLED shows “Xmitting”.

I provide RTK2go NTRIP credentials via the Base menu (3 7) (11, 13 and 14).

I exit the menu.

The survey-in restarts.

The unit connects to RTK2go using Ethernet. Corrections are sent. OLED shows “Casting”.

I use the mode button to enter “Cfg Eth” mode.

I can view the config web page over Ethernet at 192.168.0.nnn. The ZED firmware version is not shown. This is normal. Configure-Via-Ethernet requires exclusive access to SPI and Interrupts. It is not possible to communicate with the GNSS on SPI while in this mode.

I “Exit and Reset”. The unit resets and survey-in restarts. I let the survey-in complete. The unit resumes casting to RTK2go.

I use the mode button to enter “CfgWiFi” mode.

The OLED shows “RTK Config” and an IP address. I connect to RTK Config WiFi and try to open 192.168.4.1. This works. But I have seen it fail at this point.

I “Exit and Reset”.

I go back into “CfgWiFi” mode. Again it works.

I “Exit and Reset”.

I go back into “CfgWiFi” mode. This time it fails. I only get a partial web page and “Exit and Reset” has no effect.

I reset the unit using the Reset button. Survey-in begins again.

I disable Bluetooth with (s b b x x). Survey-in resumes.

I go back into “CfgWiFi” mode and again it fails.

I turn on “Heap Reporting” using (s d 1 x x x).

I see that the FreeHeap is 28624 and the HeapLowestPoint is 3972. That indicates the RAM is almost exhausted, and bad things will happen…

I reset the unit using the Reset button.

I let Survey-in complete and casting resumes.

HeapLowestPoint is 118712.

I go back into “CfgWiFi” mode. Everything works, but I see:

FreeHeap: 61936 / HeapLowestPoint: 52812 / LargestBlock: 52212
FreeHeap: 61936 / HeapLowestPoint: 52812 / LargestBlock: 52212
FreeHeap: 60636 / HeapLowestPoint: 48200 / LargestBlock: 50164
Client:192.168.4.2 /connecttest.txt
FreeHeap: 60248 / HeapLowestPoint: 47744 / LargestBlock: 48116
Client:192.168.4.2 /connecttest.txt
Client:192.168.4.2 /connecttest.txt
Client:192.168.4.2 /connecttest.txt
FreeHeap: 60480 / HeapLowestPoint: 46080 / LargestBlock: 48116
Client:192.168.4.2 /connecttest.txt
Client:192.168.4.2 /connecttest.txt
FreeHeap: 60452 / HeapLowestPoint: 46080 / LargestBlock: 48116
Client:192.168.4.2 /connecttest.txt
Client:192.168.4.2 /connecttest.txt
FreeHeap: 60480 / HeapLowestPoint: 45084 / LargestBlock: 48116
FreeHeap: 60480 / HeapLowestPoint: 45084 / LargestBlock: 48116
FreeHeap: 51788 / HeapLowestPoint: 33240 / LargestBlock: 34804
FreeHeap: 60452 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 60480 / HeapLowestPoint: 6492 / LargestBlock: 48116
FreeHeap: 59852 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 59848 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 59608 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 59608 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 59608 / HeapLowestPoint: 6492 / LargestBlock: 47092
FreeHeap: 59608 / HeapLowestPoint: 6492 / LargestBlock: 47092
Parsing: exitAndReset,1,
Reset after AP Config

I try again, and this time I see:

reeHeap: 61604 / HeapLowestPoint: 54616 / LargestBlock: 52212
FreeHeap: 61604 / HeapLowestPoint: 54616 / LargestBlock: 52212
STATE_WIFI_CONFIG_NOT_STARTED --> STATE_WIFI_CONFIG, 2025-01-15 13:32:44.187
FreeHeap: 61628 / HeapLowestPoint: 52796 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52796 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 61948 / HeapLowestPoint: 52788 / LargestBlock: 52212
FreeHeap: 60984 / HeapLowestPoint: 52508 / LargestBlock: 50164
FreeHeap: 54284 / HeapLowestPoint: 43268 / LargestBlock: 44020
FreeHeap: 54276 / HeapLowestPoint: 40500 / LargestBlock: 41972
Client:192.168.4.2 /connecttest.txt
FreeHeap: 54124 / HeapLowestPoint: 40208 / LargestBlock: 41972
FreeHeap: 52676 / HeapLowestPoint: 40208 / LargestBlock: 40948
FreeHeap: 51220 / HeapLowestPoint: 40208 / LargestBlock: 38900
FreeHeap: 49756 / HeapLowestPoint: 37732 / LargestBlock: 36852
FreeHeap: 51548 / HeapLowestPoint: 37732 / LargestBlock: 36852
FreeHeap: 49752 / HeapLowestPoint: 37732 / LargestBlock: 36852
FreeHeap: 49752 / HeapLowestPoint: 36304 / LargestBlock: 35828
FreeHeap: 49752 / HeapLowestPoint: 36304 / LargestBlock: 35828
FreeHeap: 46972 / HeapLowestPoint: 5304 / LargestBlock: 24564
FreeHeap: 46964 / HeapLowestPoint: 5304 / LargestBlock: 24564
FreeHeap: 48760 / HeapLowestPoint: 5304 / LargestBlock: 24564
FreeHeap: 50560 / HeapLowestPoint: 5304 / LargestBlock: 24564

And the Web Page fails to load…

Unfortunately, I think the evidence is pretty clear that the firmware is running out of RAM and things are failing.

I change the “Select survey-in radio” option to WiFi with (3 16).

I go back into “CfgWiFi” mode. That radio option may have helped, because this time I see:

FreeHeap: 61196 / HeapLowestPoint: 54232 / LargestBlock: 52212
FreeHeap: 61196 / HeapLowestPoint: 54232 / LargestBlock: 52212
STATE_WIFI_CONFIG_NOT_STARTED --> STATE_WIFI_CONFIG, 2025-01-15 13:42:36.129
FreeHeap: 61212 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 61532 / HeapLowestPoint: 52416 / LargestBlock: 52212
FreeHeap: 55660 / HeapLowestPoint: 44496 / LargestBlock: 46068
FreeHeap: 56796 / HeapLowestPoint: 41716 / LargestBlock: 45044
Client:192.168.4.2 /connecttest.txt
FreeHeap: 56636 / HeapLowestPoint: 41716 / LargestBlock: 45044
FreeHeap: 55188 / HeapLowestPoint: 41716 / LargestBlock: 44020
FreeHeap: 55184 / HeapLowestPoint: 41716 / LargestBlock: 44020
FreeHeap: 52856 / HeapLowestPoint: 41716 / LargestBlock: 41972
FreeHeap: 51940 / HeapLowestPoint: 11012 / LargestBlock: 39924
FreeHeap: 48264 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 49924 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48268 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48620 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48264 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48260 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48256 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48256 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 49900 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 50052 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 48576 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 49700 / HeapLowestPoint: 11012 / LargestBlock: 36852
FreeHeap: 49724 / HeapLowestPoint: 11012 / LargestBlock: 36852
Parsing: exitAndReset,1,
Reset after AP Config
newSettings len: 3413

Just for comparison, I go into “Cfg Eth” mode and I see:

FreeHeap: 94404 / HeapLowestPoint: 87508 / LargestBlock: 86004
FreeHeap: 94404 / HeapLowestPoint: 87508 / LargestBlock: 86004
STATE_CONFIG_VIA_ETH_STARTED --> STATE_CONFIG_VIA_ETH
SD card size: 14.8 GB / Free space: 14.8 GB
FreeHeap: 94404 / HeapLowestPoint: 84112 / LargestBlock: 86004
FreeHeap: 93692 / HeapLowestPoint: 84096 / LargestBlock: 83956
FreeHeap: 93708 / HeapLowestPoint: 84096 / LargestBlock: 83956
FreeHeap: 93708 / HeapLowestPoint: 84096 / LargestBlock: 83956
FreeHeap: 93708 / HeapLowestPoint: 84096 / LargestBlock: 83956
FreeHeap: 94028 / HeapLowestPoint: 84096 / LargestBlock: 86004
FreeHeap: 94028 / HeapLowestPoint: 84096 / LargestBlock: 86004
FreeHeap: 94028 / HeapLowestPoint: 84096 / LargestBlock: 86004
FreeHeap: 94028 / HeapLowestPoint: 84096 / LargestBlock: 86004
FreeHeap: 94028 / HeapLowestPoint: 84056 / LargestBlock: 86004
FreeHeap: 76064 / HeapLowestPoint: 52632 / LargestBlock: 52212
FreeHeap: 93048 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93048 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93048 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93624 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93624 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93032 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 93032 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92796 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92796 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92796 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92796 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92804 / HeapLowestPoint: 52632 / LargestBlock: 79860
FreeHeap: 92796 / HeapLowestPoint: 52632 / LargestBlock: 79860
Client:192.168.0.48 /listfiles
FreeHeap: 92116 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92108 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92116 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92108 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92116 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92108 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92116 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92108 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92116 / HeapLowestPoint: 52632 / LargestBlock: 77812
FreeHeap: 92108 / HeapLowestPoint: 52632 / LargestBlock: 77812
Parsing: exitAndReset,1,

Much healthier…!

In summary, WiFi Config is not reliable due to RAM limitations. But you should be OK with Configure-Via-Ethernet or configuring manually via the menus.

Apologies for this… This is the main reason why we migrated to the ESP32-WROVER (16MB Flash, 8MB PSRAM) on our newer RTK products.

I hope this helps,
Paul

ODMerdin · January 15, 2025, 3:32pm

Hey Paul,

Thank you for running the test and sharing your feedback. I have a few follow-up questions:

Is it possible to replace the current ESP module with an ESP32-WROVER?
Rtk2go recently banned my IP because of repeated failed connection attempts. Could that be causing the reboot issue?
I have an older device that has been working for almost a year without any problems. Would matching my firmware versions to that device help resolve the issue?

Regarding the Rtk2go ban, I only enter the caster host, caster port, mount point, and mount point password — all of which I received by email. I wonder if I’m missing some other configuration setting because, after a few tries, it looks like the device might be sending hundreds of requests with incorrect credentials. Could this be caused by a RAM or memory management issue?

Thank you again for your assistance.

Best regards,
ODM

PaulZC · January 15, 2025, 4:05pm

Hi ODM,

Sadly, no. The footprint is completely different. And you would have to rewrite the RTK Firmware to take advantage of PSRAM. The RTK EVK could be a good option for you. It has the WROVER and runs our newer RTK Everywhere Firmware. Or the RTK mosaic-X5. Same price, but runs its own excellent firmware directly on the mosaic-X5.
Unlikely. It is more likely that you were banned because of the repeated connection attempts caused by the reboots. Also, if you attempt to connect to your mountpoint as a Client when there is no Server providing corrections, that gets you banned too.
Probably?! Please try!

Apologies for the inconvenience,
Paul

ODMerdin · January 15, 2025, 6:50pm

Thank you Paul,

I will try the firmware change and get back to you if I find any other solution.

Best
ODM

PaulZC · January 17, 2025, 7:53am

Hi ODM,

Apologies. The penny has just dropped… RTK Firmware v4.1 won’t recognise the ZED HPG v1.51 firmware. Another user reported difficulties after upgrading to 1.51. The solution is to downgrade the ZED firmware to v1.50 until we get the RTK Firmware updated.

Apologies again for the inconvenience,
Paul

ODMerdin · January 17, 2025, 8:25am

Hi Paul,

We wanted to let you know that we’ve found the solution. It turned out to be a simple oversight: our RTK2go authentication wasn’t enabled. After talking with RTK2Go about the banning they enabled our authentication, after that, we successfully began streaming on RTK Station firmware 4.0, then upgraded it to 4.1 without any issues. We’re currently running RTK Station firmware 4.1 and ZED firmware 1.51.

When our IP address was banned, the system couldn’t send any data, which I assume led to an overflow error and eventually caused the device to reboot back-to-back due to insufficient RAM. We also switched to using the terminal, which has been far more effective than the Ethernet config page. I highly recommend trying the terminal if any user run into any problems with the device.

Thank you again for all your help, Paul. Hopefully, this experience will be useful to anyone facing a similar issue.

Best,
ODM

PaulZC · January 17, 2025, 9:12am

Thanks ODM - good to know,

Best,
Paul

Topic		Replies	Views
RTK Reference Station ublox firmware unknown Enclosed Products	3	140	June 20, 2024
reference station wifi not working on firmware version 4 Enclosed Products	1	320	April 18, 2024
RTK Reference Station WiFi GPS/GNSS	1	5	January 22, 2025
RTK Reference station - bumpy but up and running correctly? Enclosed Products	4	45	January 16, 2025
RTK Reference Station - Boot Loop Enclosed Products	8	256	July 1, 2024

RTK Reference Station Reboots

Related topics