Linux

The reality is what youre asking for is very complex - you're asking for lagless streaming for a desktop. That is running a GUI on remote hardware, and then streaming that video to another computer with low latency so you have no perception of lag in moving the mouseor interaction, and continuous streaming of desktop updates.

There are lots of factors at play that can make it a poor experience.

You can have what you want if:

• The server you SSH in to has the resources to run X well
• The server you SSH in to has the hardware to be able to then convert that to video (with some tricks) and stream it
• The internet connection between you and the remote server is stable and high enough bandwidth to stream the desktop
• the internet connection between you and the remote desktop is low latency.

Its very hard to achieve all those things even when youre creating machines that are dedicated for remote desktop streaming. I have done that in my work with Windows devices and to get good quality streaming we needed dedicated hardware, dedicated software and high quality internet. And even then some of our users had bad experiences.

Most remote servers are definitely not set up to provide what you want. Dedicated software for the task will help as there are lits of tricks that they apply to make a streaming desktop appear latency free versus simpler solutions that just stream the actual desktop.

VNC is not a good solution - its basically just taking screenshots and streaming those to you. It works with fast devices on a local network, but is very limited in your use.

If you really want to solve this look at software optimised for low latency uses such as gaming. For example Moonlight/Sunshine are for game streaming but work with desktops. They are designed to be low latency high quality. But to achieve that you need the video hardware on your server, and the good low latency stable internet connection.

Real world high quality desktop streaming also needs good graphics hardware and optimised tools. It can be achieved with open source software but you need the hardware to to do the heavy lifting.

xrdp and x11vnc is rootless

Yes that's to much to ask

You are adding a ton of overhead and complexity. Expecting it to just work is underestimating the difficulty of it all. With that being said, Xpra is probably what you are looking for. However, I would suggest that you take a look at your use case as there is probably a better solution. What app are you trying to use?

Also, Xorg is a bit of a dinosaur at this point. It isn't going away but gnome and kde have both made plans to drop support long term. We still have a few years but from a longevity perspective I would start thinking about new solutions. KDE and Gnome both support RDP and there are plenty of modern web interfaces for managing servers like Cockpit.

Im searching for a solution too. You mentioned wprs, have you tried its predecessor xpra?

And theres always stuff like teamviever, rustdesk, and my personal favorite moonlight/sunshine, although all of these are for a slightly different usecase(full access to a remote desktop).

i just use sunshine if i want to use gui apps. its annoying to setup in a headless server but it works.

the feasibility of this solution will depend on your usecase.

I normally just use X forwarding over ssh. For simple, X native, apps it usually works nicely, but if you're using something that draws its own UI (electron apps, browsers and the like often do) it'll be extremely painful or just fail.

I believe there is, or was, a way to run a vnc client rootless, but I think you needed to configure the server in a specific way too. It's been a long time since I tried though so things will have changed.

This might be dumb, but what if you used parsec on the other machine?

Kasmvnc?