If you have a remote server with UI, you can set up a X Window server on your Windows/MacOS machine and forward via SSH X messages to use GUI apps on your server, but view the result locally. The Responsiveness of the UI depends on your network capabilities.
This is fun and all, but if you lose your connection, your windows will go away and your program will usually exit.
It’s almost always more usable to run a desktop session in Xvnc on the server and connect to it with a VNC client, because if you get disconnected, you can just reconnect.
VNC is also much faster in most cases, because the X protocol requires a lot of round trips that waste a lot of time. So instead the VNC server "runs these round-trips locally" (so to speak) and you only interact with the pixels remotely. X was developed when bandwidth was as limiting as latency but nowadays only latency is limiting so a different protocol makes more sense.
You can't pipeline X11 operations in the presence of anything but a perfect network because
1. TCP streams require stalling when packets are dropped to keep the stream in order
2. X _requires_ by design that commands are performed in order
Which means that using something that can do UDP and manages it's own sequence ordering can do significantly better. This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years ago and you still can't with X protocol over the network.
Breaking up the screen into small 16x16 chunks or so, encoding on the GPU, and shipping that turns out to be significantly faster.
Especially when you take into account that virtually _nothing_ draws with X using X drawing primitives. It's almost all using Xshm for anything non-trivial.
Most modern Ethernet LANs are effectively lossless. You're not going to see a single dropped packet unless you saturate the link.
> Which means that using something that can do UDP and manages it's own sequence ordering can do significantly better. This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years
No it isn't. It's because those things actually compress the video, and X-forwarding generally doesn't. The transport protocol is completely irrelevant, it's just a bandwidth problem.
I've X-forwarded Firefox between two desktops on 10G Ethernet. I can watch 4K video, and I genuinely can't tell the difference between local and remote.
If you're including TCP ACKs as part of the "chatty"/"required round trips" of a higher level protocol, that's bad new for a lot of things. (Which, granted, is why they made those QUIC protocols etc., but still, it seems unreasonable to single out X's protocol for this, especially since RDP and VNC are commonly used over TCP as well).
But:
> This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years ago and you still can't with X protocol over the network.
Compression is going to have a much bigger impact over a large motion than most anything else; you can stream video over HTTP 1.1 / TCP thanks to video codecs, but X (sadly i think, seems like such an easy thing that should have been in an extension, but even png or jpeg never made it in) doesn't support any of that.
> It's almost all using Xshm for anything non-trivial.
Xshm is not available over a network link and it is common for client applications to detect this and gracefully degrade.
VNC hasn’t always been that good though. There was a time when I’d routinely use X forwarding via SSH because my (then) girlfriends internet connection wasn’t fast enough to run VNC smoothly.
different implementations of vnc support different compression algorithms and for some reason vnc always defaults to the worst (most compatible?!?) ones.
You can achieve a lot by tweaking the client and server settings.
The results were good enought for me to support customer with ISDN internet between 2001-2006.
or you are like me and remote-desktop from a thin client (or from home) to a windows VDI, run vnc to a X server and ssh -X to the dev box to run emacs (edit: and firefox). I do this every day and I'm amazed that it actually works smoothly.
Same deal with a remote shell. Usually it's not a problem, but if you anticipate it will be, you can use additional tools (screen, tmux, Xpra).
Xvnc is fine if you want a desktop session, but usually I just want one program, so X11 is a better fit for me. Once you have the infrastructure setup, it's really easy to go from a normal shell to running a graphical program. For me, I use a Microsoft Windows desktop environment, have X11 forwarding enabled on my usual shell targets, so I just start the X server and run the program. I could have the X server autostart (it's unobtrusive if there's no clients), but I haven't yet. On a mac, the Xserver starts automagically (once installed), so it's just a matter of enabling X forwarding in your ssh config to hosts where it's likely. Even less work if you run an X desktop.
> your windows will go away and your program will usually exit.
This is a huge improvement over most "web applications", whereby your client (web browser) sits there looking like it's doing something when your network connection goes away. Worse still, you don't know whether the last (trans)action you sent to the server was processed or not.
> This is fun and all, but if you lose your connection, your windows will go away and your program will usually exit.
Interestingly, that is xlib's behavior moreso than inherent in the protocol. xlib assumes connection lost is a fatal event, but if you're doing your own socket, you can choose to do this differently. (or, even with xlib, you can throw an exception from the connection lost callback and regain control before letting it abort)
Some server state is lost, but it is possible, with some care, to recreate that from your client upon reconnecting. You can even connect to a different server and somewhat seamlessly migrate windows over.
You can survives disconnects even with xlib, I think the abort is just the default behavior.
Emacs is the one program I know of that can actually do this. It can pop up frames (Emacs lingo for windows) on multiple displays at the same time and even mix tty and X11 frames. The Emacs session survives connection loss fine, frames on other displays continue working and you can reconnect if desired.
The one caveat is that Gtk used to have a bug that caused it to uncontrollably abort on connection loss but I build my Emacs with --without-x-toolkit (so it uses raw xlib, no toolkit) and that configuration has always been robust and performant. If I remember correctly the Gtk bug might be fixed now too.
Sure, although originally you'd have an X Terminal (an actual dedicated terminal device running an X server, of varying levels of performance), rather than an X server program running on your computer (i.e. an X Terminal emulator).
The terminology is of course a bit counter-intuitive, since it's program-centric rather than computer-centric - the local terminal is the server and the remote computer (or rather program) is the client, utilizing the server as a display device.
It's a bit like a text terminal (VT-100 etc) except of course the X terminal has a network connection., With the VT-100 if you wanted to connect to a remote system you'd have to use a modem (acoustic coupler) to dial into a terminal server on the remote system. I don't think anyone ever made a text terminal with built-in telnet and ethernet.
For some reason I always found it fun messing with serial cables, and always had a desk draw full of various types (cross-over, gender-benders, db9-db25 etc), as well as occasionally needing to make custom ones.
I still have a USB terminal server for when I want to play with old stuff like my DECTalk (VCR-sized speech synth) which has two DB-25 serial ports on it.
I did the same thing! I actually got my first soldering iron (and learned how to use it) so that I could wire to my own connectors. Building a home-made graph-link cable for my TI-83 in particular was a fun experience, as well as a good education in what can go wrong when you mess up soldering.
Not telnet/ethernet, but later VTs had built-in modem controls and some utilities in firmware, so you could have an AT-capable modem connected over serial port and just select what connection to dial into.
I used to do this with Virtual Box running locally. I found the performance of the windows x client was superior to trying to use x directly in the virtual machine. I wish the protocol supported audio.
I used to run everything over 100Mbps ethernet. It worked just fine. Two biggest issues: Ssh without tuning settings, and modern X applications increasingly doing client-side rendering into bitmaps and so being far more bandwidth hungry and not tested over the network. If it's the latter, VNC or XPRA are much better suited if what the app ends up doing is streaming bitmaps anyway, as they're actually optimised for that.
It’s usually SSH that’s the bottleneck though. If you just expose your local X11 TCP server port and connect to it from the server (X11 client) by setting the DISPLAY env properly, it will be much more responsive. Often indistinguishable from local apps. Secure the port with Wireguard, if you want to be a bit more responsible and still have good performance. Checked with Firefox, which works like a charm this way, but is unusable over SSH.
> If you just expose your local X11 TCP server port and connect to it from the server (X11 client) by setting the DISPLAY env properly, it will be much more responsive.
That's fine on a LAN with low or sub ms latency. If you need to connect to a server on the other side of the country, you'll want a less chatty protocol, e.g. X2GO (which gives additional benefits, e.g. of restartable sessions).
The catch is latency. In my case both hosts are on the same colo.
I remember, 20+ years ago running quake3 over remote X to my roommate machine and I was surprised that glx actually worked over the network with no issues.
It depends on how the client was written. If you build around asynchronous operations, you can get good throughput, and if you always stop and wait for a reply, you'll get bad throughput. If you buffer your output and don't actually send it for a while, that can also make the experience poor. Xlib pushed you in both those directions, but you can make terrible clients with any library.
My personal experience with Oracle products has always involved terrible UX, so I'm not surprised they built a terrible X client.
I've run browsers over X, sometimes actually remotely, and it's OK, but not great. I wouldn't recommend it for video, audio is out of scope, and start up can be very slow sometimes. It's something that used to work better, IMHO. Sometimes you need to try a couple browsers and see which one works better. As a sibling notes, it all feels untested, and I'm sure if developers were testing, they'd fix some of the things that are easier to fix.
X over SSH works really well for me as long as I enable SSH compression. At least for my usage, I can't tell it apart from a native window but then again I'm not running sophisticated programs which update frequently.
Until fairly recently I used to use Quartus on a headless box with ssh -X
It worked really well up until a certain version (18, I think) - after which something changed in the UI toolkit and it was suddenly much more laggy. I'm guessing the UI toolkit's newer incarnation required a lot more round trips.
x2go is a bit smarter than pure x-forwardning and also allows you to resume a session at a different time (or on a different machine) similar to something like tmux. Much much better performance and does pretty well even on cellular.
A bit rough around the edges but brilliant for some usecases.
The use case was a whole bunch of cheap display server machines in a computer lab at a university with a very powerful machine running the computation-expensive client programs. You didn't need SSH for that and the only real problem was contention on the token ring (ah, the old days...)
If you have a remote server with UI, you can set up a X Window server on your Windows/MacOS machine and forward via SSH X messages to use GUI apps on your server, but view the result locally. The Responsiveness of the UI depends on your network capabilities.