I was trying out FSR4 on my RX 6800 XT, Fedora 42. Works really well and it easily beats FSR3 in visuals even on Performance. It does have a significant performance hit vs FSR3 though but it still works out to be a bit faster than a native rendering on Quality.
I was trying to explain why the game loop would be held back by the rendering speed, even though they run on different hardware.
If you are bottlenecked by the GPU that means the game loop spends some of its time waiting for the GPU. If you then turn on frame generation, you devote parts of the GPU to doing that, which makes regular rendering slower, making the game loop spend even more time waiting. This will increase input latency.
Frame generation also needs to delay output of any real frame while it creates and inserts a generated frame. This will add some output latency as well.
In the opposite scenario, where you are bottlenecked by the CPU, enabling frame generation should in theory not impact the game loop at all. In that case it’s the GPU that’s waiting for the CPU, and it can use some of those extra resources it has to do frame generation with no impact on input latency.