Seems like a crowd favorite, previous discussions here:

2012: https://news.ycombinator.com/item?id=4062981 2012: https://news.ycombinator.com/item?id=4544318 2014: https://news.ycombinator.com/item?id=8094186 2015: https://news.ycombinator.com/item?id=10064657 2015: https://news.ycombinator.com/item?id=10631513 2017: https://news.ycombinator.com/item?id=13571055

I think it's less the subject matter and more the depth and quality of the article on the subject.

Anyone work with TTYs much? It's something that comes up from time to time and I think I always come away with a 70% understanding.

Say I have a server process and I want to be able to netcat onto an open port and have it expose a shell like experience. History, line editing, scrollback etc. I know ssh accomplishes it somehow, but it gets a local and remote process in between me an the remote bash process to set up some tty/pty magic.

What would I need to do to get something similar working?

History, line editing and scrollback are provided by readline - at least for bash.

http://web.mit.edu/gnu/doc/html/rlman_2.html

Instead of readline one can also use libedit; the main difference being that libedit is BSD-licensed, while readline is GPL - not LGPL. That means you cannot really use readline with software that is Open Source, but not GPL-ed. LinuxCNC has (or used to have) this problem, for example.

The question is always how do I get it to read/write to a socket (in process) instead of assuming I want to do it on STDIN/OUT? I didn't see an API for it.

Clearing a confusion: It's always stdin/stdout, but you can change what stdin/stdout are. E.g. when you do 'cat < foo.txt > bar.txt' you don't change the fd numbers that cat reads from, (they still read from stdin/stdout a.k.a fds 0 and 1), but you change what they represent (in this case, they represent opened text files). That opening of files happens before cat's process image is even loaded.

Same goes for bash, it's just a program reading from stdin/stdout. There's a slight difference though, in case of interactive applications like bash: stdin/stdout would better be a terminal (e.g. /dev/tty* or /dev/pty* ). That's because a shell emits, besides textual output, control codes for interpretation by the terminal. These control codes are not displayed as readable text by the terminal. Instead, they do things like "clear the screen", "position the cursor at x/y", or "set the foreground color to green".

Now, you could have a netcat between a local terminal and a shell running on a remote server. But the problem there is that the shell is not blindly reading/writing from/to an input/output stream pair. A terminal is also an entity that's deeply connected to the Operating system and job control. That's why the terminal really has to "run" on the same machine as the shell.

So this is what ssh does: It connects to the remote server, and sets up a pseudo terminal there, so that the remote shell can interact with it and the operating system. On the local side, interpretation of control codes and job control by the local terminal are disabled while the remote system is running (the local terminal is put into so-called raw mode). For example, when you press Ctrl+z (ASCII 26), this does not result in a process suspend on the local side anymore. Instead the byte (26 aka 0x1a) is sent to the remote server, and only there it is interpreted by the terminal subsystem in the operating system, to suspend the current remote operation.

It's probably not super clear from my description, so you should just play with it a little. For example, do "printf 'sleep 10\n\x1a\necho hello\nexit\n' | ssh -t -t jstimpfle.de". What's happening here?

Actually, shells do not just use standard input and standard output, albeit that (in the interactive case) the other file descriptors are usually pointing to terminal devices too.

* https://unix.stackexchange.com/a/434839/5132

Yep, that's true. I was missing some finer details. There is stderr, which usually is the same as stdout (terminal), and you can run shells in noninteractive mode (they will detect when they are not connected to a terminal). In this case there is no prompt and no job control.

One possibility is to carefully arrange file descriptors manually, so that fds 0 and 1/2 go to the socket. Check out dup2(2).

But that's not probably what you want, or sensible.

Yea that's a no go. The rest of the process is doing stuff with those!

GNU readline gives the option to provide custom stdio streams.

rl_{in,out}stream. You can use fdopen to get a stdio stream from a fd (socket for example). This can be problematic if not done correctly (buffering & stuff). Official documentation has more info on all this.

You can also fork+dup2+pipe/pty and use stdin/out on the forked process.