Rclone syncs your files to cloud storage - https://news.ycombinator.com/item?id=29213024 - Nov 2021 (8 comments)

Rclone – Sync files and directories to many cloud storage providers - https://news.ycombinator.com/item?id=22791036 - April 2020 (100 comments)

Rclone – Rsync for Cloud Storage - https://news.ycombinator.com/item?id=21906779 - Dec 2019 (3 comments)

Rclone: rsync for cloud storage - https://news.ycombinator.com/item?id=12398303 - Aug 2016 (129 comments)

Rclone: rsync for cloud storage - https://news.ycombinator.com/item?id=12049507 - July 2016 (1 comment)

Rclone – rsync for cloud storage - https://news.ycombinator.com/item?id=12045729 - July 2016 (3 comments)

Rclone – rsync for cloud storage - https://news.ycombinator.com/item?id=11404178 - April 2016 (1 comment)

Because:

* Rclone offers client side encryption (with a good algorithm and from a good library). Cloud providers usually offer server side encryption, but don’t want to offer client side encryption.

* You can mount the remote. Files don’t have to exist on client’s side to occupy space. This tool mounts anything!

* You learn one tool and use it everywhere

* It’s is under user’s control. You run it with a schedule that you want. I don’t have to run a closed source app continuously scanning my computer

* Robust, feature full, open source (especially for verification of encryption), large community for getting help, and under heavy development Rclone combined with Restic is how I manage my data in the cloud.

A few years back, around 2016 I wanted to see how One drive syncing worked. So I set my IRC client (hexchat) to save logs on my OneDrive folder. If I remember correctly, I had hundreds of gigabytes uploaded to onedrive under a month. I think I had the default 15 GB or 25 GB storage iirc. My guess is every time someone said something in a chat, the file changed and one drive thought it was prudent to sync it with its servers immediately?

When a file changes, should we upload the entire file? Is it possible to somehow only transmit the diff and only send what has changed? Would it even make sense as most "normal" users are probably on mobile and diff has a cost in CPU and therefore battery, right? Or is CPU cheaper than WiFi for battery?

One of the features I like about Amazon.com Photos on Android is I can say only upload photos when charging.

This is what rsync does.

edit this refers to rsync the software, not rsync the company that provides storage.

Can you view files in Dropbox website if they are encrypted locally by Rclone?

> You can mount the remote.

Do you have to mount the remote every time after rebooting your machine?

If I use Rclone to sync all dropbox files to my local machine, are they still accessible if my machine is offline?

context : first hand experience in IR

What would be my preferred VPN/Mail/Payment/Storage/Encryption tools?

The one that has been scrutinized by TLA to catch these groups. On a long enough time scale, it can expose if they truly are "no logs", "fully encrypted", "privacy-conscious" or if it's just marketing buzzwords when they actually face a warrant.

Plus rclone makes it easy to migrate between providers. I just moved a backup process from Gcloud to B2 now my side business is growing up: you can start with whatever storage is free/convenient then switch to something more mature later with minimal effort.

<blinks>

Holy shit. I didn't know that was an option. I just assumed that there was some minimum number of users (like 10-20) that made it non-viable for personal use-cases.

Been personally using for 1.5 years with Backblaze thanks to HN with 0 problems. Have even done some restores succesfully!

https://rsync.net/resources/howto/rclone.html

and although the example is S3 <-> rsync.net, it is not necessarily rsync.net-specific.

You could use this HOWTO/workflow for any two targets.

    4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
      \ "s3"

The rclone docs are very extensive but having a short, simple example was the kick I needed to get past the myriad options.

[1]: https://rclone.org/overview/

[2]: https://rclone.org/crypt/

[3]: https://rclone.org/gui/

[0] https://github.com/rclone/rclone/tree/master/librclone

What I like most about B2 is the transparent, no gimmick pricing. No monthly minimum, no minimum file size, no paying for storage for 90 days after you delete a file. And they give generous free allowances: 10GB/mo (forever, with no credit card required), 1GB of free egress per day, and up to 5000 free transactions per day.

I backup the HashBackup dev server to B2 every night, around 80GB with mostly VM images, and the total backup size while keeping the last 30 days + 12 monthly backups is around 50 GB and costs 25 cents/month. They bill me every few months, I guess when enough charges accumulate.

Here are the backup stats for the HashBackup dev server:

[jim@bs ~]$ hb stats -c /hbbackup HashBackup #2527 Copyright 2009-2021 HashBackup, LLC Backup directory: /hbbackup

               3,617 backups
              334 TB file bytes checked since initial backup
               80 TB file bytes saved since initial backup
                 910 total backup hours
               79 GB average file bytes checked per backup in last 5 backups
               23 GB average file bytes saved per backup in last 5 backups
              29.78% average changed data percentage per backup in last 5 backups
             13m 43s average backup time for last 5 backups
                 637 archives
               55 GB archive space
              84.11% archive space utilization 46 GB
               810:1 industry standard dedup ratio
              9.5 MB average archive space per backup for last 5 backups
              2465:1 reduction ratio of backed up files for last 5 backups
              234 MB dedup table current size
           9,442,920 dedup table entries
                 48% dedup table utilization at current size
             724,059 files
             671,701 paths
          10,550,610 blocks
         127,965,672 block references
         117,415,062 deduped blocks
              12.1:1 block dedup ratio
              5.2 KB average stored block size
              616 GB backup space saved by dedup
              49,648 average variable-block length

  ssh user@rsync.net rclone sync s3remote:bucket path/path/dir

In fact, you don't even need to install rclone yourself - just run it, over SSH, at rsync.net.

Here is a broad overview of using rclone with your rsync.net account:

https://www.rsync.net/products/rclone.html

Your prices aren't offensive, but they're not cheap.

I have a need to store several terabytes of PDFs - mainly old magazines that are in the public domain, or where the copyright is essentially lost. Are your systems going to scan every item that gets uploaded and then erase my account and all my files if one PDF comes up as having a current copyright issue?

I ask in all seriousness because this happens regularly to those of us on

https://www.reddit.com/r/DataHoarder/

see, e.g.

https://www.reddit.com/r/Degoo/comments/iixlse/degoo_copyrig...

This is who we are:

https://www.rsync.net/resources/notices/canary.txt

We were the first provider to publish a Warrant Canary and it appears we're one of the few that still update it.

So, no.

No, we do not scan accounts for any purpose whatsoever.

Please review our extremely short and simple privacy policy:

https://www.rsync.net/resources/regulatory/privacy.html

... and do be in touch about the HN readers discount.

EDIT: rsync.net has, in the past, donated a lot of resources to Jason Scott and Team Archive for archival purposes just like you are describing. I am also, personally, a sustaining donor of Internet Archive. All of this to say: I am very interested in this archival work you are doing and would like to help support it - please do be in touch ...

> In September 2014, U.S. security researcher Moxie Marlinspike wrote that "every lawyer I've spoken to has indicated that having a 'canary' you remove or choose not to update would likely have the same legal consequences as simply posting something that explicitly says you've received something."

> ..That said, case law specific to the United States would render the covert continuance of warrant canaries subject to constitutionality challenges.[citation needed] West Virginia State Board of Education v. Barnette and Wooley v. Maynard rule the Free Speech Clause prohibits compelling someone to speak against one's wishes; this can easily be extended to prevent someone from being compelled to lie.

https://en.wikipedia.org/wiki/Warrant_canary

> https://www.rsync.net/resources/regulatory/privacy.html

It says there that you don't use cookies, but to clarify, I assume you set a cookie when a user logs in?

It has been described as "The Holy Grail" of backups:

https://www.stavros.io/posts/holy-grail-backups/

I am looking for another backup tool that backs up to an S3 storage destination and has a decent GUI (restic doesn’t have a GUI; relica doesn’t count; and borg doesn’t support s3). Since rclone has an experimental GUI I wondered.

For personal file backup, you only need a fairly reliable object storage system. And Backblaze seems to be the cheapest.

We have had a "HN reader discount" for many years ... just email and ask.

If you're optimizing for price, I would suggest Backblaze.

Hetzner storage, which is comparable to your offerings (supporting a dozen or so methods), is 9.4 euros per month per TB.

How do you justify being so much more expensive?

There’s a .info that shows up in search results that I think is an otherwise unaffiliated reseller that they haven’t bothered to go after.

By the way they seem to offer quite some storage for free! :-o How do they sustain? and even paid prices are unreal.

Also I failed to understand whether they’re a file sharing service or file backup?

To be clear, is the $25/TB/month quoted with or without the HN discount?

The HN discount price is lower than that (by a fair amount).

The downsides to the discounted plans are as follows:

- you can't have subaccounts/sublogins (child accounts).

- we won't give you technical support for borg/attic/rclone integrations. We assume you are an expert. Of course you get normal run of the mill support ...

For example if you have sufficient need for Microsoft Office programs to justify a Microsoft 365 subscription, that comes with 1 TB of OneDrive on the Personal subscription ($6.99/month, $69.99/year).

Most backup programs seem to know how to work with OneDrive directly or know how to use rclone which knows how to work with OneDrive [1].

If you need more than 1 TB, OneDrive becomes less attractive. It seems you can add another 1 TB for $9.99/month, but unless you have to all your storage be in one place, it would be cheaper to get your second TB at Backblaze. (And if you need 2 TB in one one place, buying 2 TB at Backblaze + An MS 365 Personal subscription would be the same price as an MS 365 Personal + 1 TB Extra OneDrive, and the former gives you your 2 TB in one place at BB and 1 TB at OneDrive).

The Family subscription ($9.99/month or $99.99/year) includes 6 TB of OneDrive, but it is partitioned into 6 separate 1 TB shares for 6 people on the account. A person is identified by an email address, so there is nothing stopping you from making 5 fake people, and one person can give another write access to folders on their OneDrive, so you could cobble together a scheme to use all 6 TB but there would be some hassle.

I tried doing that once, having the fake person give the real me write access to a folder. I then uploaded a bunch of files to that shared folder from my regular account. Shortly afterward the fake person got disabled for TOS violations (but could be reenabled via a simple automated process). I'm not sure what the issue was, but what I uploaded was a bunch of Springer ebooks to from the large collection that Springer made freely downloadable early during COVID so that I could read them on my iPad. It could be that triggered some sort of anti-piracy measures.

If you can find a way to make it work, though, 6 TB for $9.99/month or $99.year is a good deal even if you don't need the Office apps.

[1] https://rclone.org/onedrive/

There is no need for your backup provider to be able to look into your files so there’s only downsides to allowing them to.

Seems rsync.net supports this (see sibling comment). Any others?

I'd offer a warning that I have used B2 for highly available blob storage before (2-3 years ago) ... it didn't work well as traffic scaled. I'm not sure if that's the case now.

For anyone wondering about B2 encryption. I looked at B2 last year and ultimately didn't use it as they didn't have encryption at rest [1]. It seems they've now added that [2].

[1]: Client-side encryption would have needed quite a bit of engineering to make work for my use case.

[2]: https://www.backblaze.com/b2/docs/server_side_encryption.htm...

I don't really know how this misunderstanding is so pervasive, even among the HN crowd, but no, no amount of magical code can make it possible to directly migrate between S3 and Google Drive. It's going to download, and upload to sync.

This is even reflected in the project's description line, which is more accurate than the current HN headline: Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

That having been said, it's of course invaluable software. Mounting 10TB of Google Drive into a Raspberry Pi running Plex is still something that feels like it should be harder to accomplish than it is.

writes magical code that fires up an unmetered-bandwidth billed-hourly VPS for a few hours (Scaleway is nice)

Using Google Colab is the best way I've found so far - mount your GDrive and you can wget to it directly from Dropbox (or anything else).