Qwiicbus and pullups

I have three devices on my I2C bus, reading at a fairly high rate (bus is at 400KhZ). When all three are plugged in, I get random data on the first device (ICM20948 IMU). I’ve read that the solution to this is to disable pullups on some devices to reduce total resistance.

However, the connection will change to:
ESP32 <-qwiic-> IMU <-qwiicBUS-> sensor2 <-qwiic-> sensor3

The other 2 sensors are a Hall effect module and a BNO086 module, and they will be about 2 meters from the MCU/IMU.

Question: does QwiicBus change the pullup resistance on the I2C bus? Should I remove some pullup resistors? If so, which ones would be best?

Remove all but one set on any given single i2c bus