rsync over ssh works perfectly (i.e. with the rsync option --rsh="ssh").
Not only it works much faster than scp or sftp, but it also does not lose any file metadata.
I have not verified if newer ssh versions have corrected this, but a few years ago scp and sftp failed to copy extended attributes and on some file systems they truncated timestamps.
I'm probably living in my own little bubble, but I didn't know anyone used sftp for ANYTHING, at least not in the last decade.
Looking at my terminal history, scp for a single file, rsync (which you don't need to pass that --rsh flag to.. during the 21st century ;P) for anything that requires the ability to be interrupted and resume gracefully (which is most cases), and, in the rare case when I need speed above all, the "tar lzop ssh bash pipe" trick (I don't know if it has a canonical name) is correct.
Isn't this what we all do?!? If there's a better way, I don't see how, but would like to know.
I use sftp every now and then I need to pull something and I don't know where exactly that is. A shortcut for finding the path via ssh and then pulling the file via scp, if you will.
In that rare situation, I typically use tab completion to find the remote file. However, I bet that would SUCK on a high latency or low bandwidth link, where sftp would be just fine.
Note that sftp != ftps. sftp is scp. Rsync is better if you can get it, but I'm pretty sure that whenever I transfer stuff using WinSCP and File Explorer (Android) it's using sftp.
sftp still used extensively in banking, payments, transferring financial data, or datasets like student information from school districts or information on lawyers from bar associations.
Back in 1999, in one of my first professional gigs, people were still using ftp for sensitive data for automated data transfers.
Rsync+ssh would be great, but enterprise IT is slow to change.
AWS has hosted sftp backed by S3, and it's going to stick around for a while.
Sadly in 2021, ftp is still used extensively in financial services for stuff like clearing stock trades. I wish more institutions use sftp/scp for this and stop relying on allowing specific IP addresses for security.
> I didn't know anyone used sftp for ANYTHING, at least not in the last decade.
We're moving customers from FTP to SFTP. Most recent one was a well-known Fortune 500 company, which uses it to exchange some ASCII flat files with our software.
Mostly it's for XML files or PDFs.
It's simple, it works and it's easy for support to poke and prod.
You raise a good point, and I was incorrect. In my original comment I was thinking of any of us, not muggles.
Come to think of it, I once even set it up at a job in 2009. We needed to digest huge amounts of flat files from clients with little tech savvy. I put FTP on a server, put that behind OpenVPN, then sent the customers a pre-configured OpenVPN client installer. Never broke.
Tangentially, all this mention of FTP being used by banks and financial institutions is inspiring some choice nmap scans to run... ;)
Like someone else said sftp it’s huge for B2B and probably is part of how you get paid.
*ftp is blunt but nice when dealing with other parties since it interoperates well, everyone has it, and (most of the time) it just works whether you are talking to a Linux box, IBM MQ, or something else altogether.
Solid Explorer (Android app) connects via sftp to Linux machines just fine. While I use Termux and rsync to do backups and bulk transfers, Solid Explorer allows me to rapidly browse remote files.
I use rsync+ssh a lot, and it doesn't exactly work perfectly on long-distance 10-100 gigabit networks: ssh has a buffering problem at high bandwidths which the maintainers won't accept patches to fix.
What back-end do you use? Also on top of that back-end do use type = crypt, directory_name_encryption = true and so on?
I tied various options and found rclone were slow for me.
This is interesting, can this be the reason for why playing 720p video in Firefox run with ssh and X11 forwarding is fine bandwidth-wise (easily 1Gb/s) but it causes responsiveness to mouse/keyboard actions to go into abysmal 5-30s territory?
Any references to the ssh buffering problem out there?
I understand and sympathize with rsync advocacy, but there are two major problems here.
The first is the license. Because rsync is under the GPL, it will never be bundled in the OpenSSH.com distribution. It is even missing from OpenBSD base (just confirmed, and loaded it with pkg_add - there are two versions, one with iconv).
So let's take Microsoft's implementation, which will forever lack rsync (as they haven't touched it in well over a year).
The file transfer agent needs to be under the same license.
rsync is a different program than ssh, installed as a different package, so it does not matter what license it has. In most cases, ssh will run a bash at the remote end, which also has the GPL license, so it should be obvious that it is irrelevant what license have the programs launched by ssh.
When you run rsync over ssh, the local rsync starts the local ssh client, which connects to the ssh server, which runs the remote rsync, while the data is piped through the ssh connection.
rsync is available for most operating systems, but it is not usually installed by default. OpenBSD might be the only exception without a rsync package, but it has openrsync. I assume that it uses the same protocol so rsync to/from OpenBSD should work as well as to/from Linux, Windows, MacOS or any *BSD.
If you have the right to transfer files to a computer then normally it is very simple to install rsync, only for your own user if do not have admin rights.
In most cases, you can initially just copy the rsync executable file to the target computer using scp, and then do any other file transfers with rsync over ssh.
After I have abandoned both scp and sftp many years ago, I no longer had any problems with the performance or data integrity of file transfers over ssh.
Not only it works much faster than scp or sftp, but it also does not lose any file metadata.
I have not verified if newer ssh versions have corrected this, but a few years ago scp and sftp failed to copy extended attributes and on some file systems they truncated timestamps.