I can only hear my REX at very low speeds and even then I don't find the noise objectionable. Maybe there is something wrong with your exhaust system. Or maybe my hearing is bad.
The noise might not be from the REX engine but from the cooling fan. The engine is supposed to stop at low speeds but, in my experience, the fan will continue for some time after stopping. If the fan is out of balance that would make the noise worse.
I have a button set for the Hold-SoC feature so I can turn it on and off easily. Maybe it would help to turn off the Hold when you are in town and likely to be stopping often.