I went into some detail about load cells last week:
The one linked appears poorly suited for wet environments for all of the reasons you’ve guessed. The exposed wires are bad enough but, under the white glue, the gauges on the load cell surface are very vulnerable to contamination and corrosion and delaminating and need to be protected. It will probably be maybe okish for a while, some hours? Days? On the other, more positive hand, these load cells are inexpensive enough to try & see and, in clean STP water, might actually last longer than any of us think. But you have to assume that water will get in there and potentially affect weight results in unpredictable ways.
They do manufacture fully submersible load cells for weighing and force measurement at oceanic depth but, more commonly, highly water resistant ones for wet (but not immersed beyond 1 meter) environments where scales are used a LOT like commercial food production, pharma mfg, outdoors, etc. The gauges are hermetically sealed within the body of the cell and the cable entry is tightly closed with cordgrips or swaging. Care must be taken to protect cables which, if nicked, can wick water into the load cell or instrument. Research terms: washdown load cell, IP67, IP69k, NEMA4