For some games I'm happy to turn it on as it drops power/temps and provides virtually identical output (as far as i can tell anyway) to native, but my fans don't need to go into overdrive mode.
I may even put on frame gen too if I just want to bump a stable-ish 80-90fps to stable 120fps, and again drops power and temps slightly. That sometimes does cause smearing but for the most part I don't notice enough to be annoyed. Without them I would probably be running with more power draw and higher temps, and possibly still not even hitting lower resolutions at 120, some games as you say though can hit 120 no problem even without and the gpu won't be stressed.
In some games it's super obvious that it's doing upscaling and looks awful, some games with without upscaling look awful (TAA), it's odd that in some cases tools like Dlss can look better than native TAA.
For the most part though it feels like the FLAC vs Opus or whatever, most people can't notice difference and don't care enough about it, but some people can tell the difference and want the best, I don't think either are wrong it's just down to how much you notice/can tolerate before it's annoying.