I used to seriously stress out worrying about the huge amount of family photos I have saved in cloud storage and what it would mean to lose them. I've got a local backup as well of course. Even that has to be upgraded every so often to make sure the hardware will continue to run reliably.
A few years ago my (very non-technical) mother asked me to make and send her a photo album of recent pictures. My wife and I had so much fun doing it that we kept doing it. Choosing the best pictures, arranging them and making nice layouts and titles itself is a fun walk down memory lane.
Now we have a shelf with about a dozen volumes of memories and we found we are far more likely to look at these than we ever were with photos in cloud storage.
I like to think that if I lost all the digital copies, I would still be happy with what I have in these bound albums. I highly recommend this to everyone.
2) find a Mac with a very large system drive (1tb or above)
3) open the photos app on mac and ensure settings is configured to “download originals” not “save space”
4) make sure the computer is online on a regular basis. You should see ~/Pictures/Photos.app becoming massive over time.
5) setup Time Machine for that computer
6) success! You have a second copy of your photos library on Time Machine now.
7) Use Arq.app to backup your Time Machine drive to AWS glacier. Nice! Now you have 3 copies of your library.
An alternative path is making sure you have both a giant iPhone storage and giant Mac storage. Make sure photos on phone is set to “download originals”. Manually backup your iPhone to Mac monthly. These backups include photos library. I recommend encrypting your backup and keeping a key in safe and 1Password. Continue with Time Machine and Arq.
That's a nice comprehensive approach. I'm actually pretty happy with my setup from a data loss prevention perspective. It is just that, as I approach 2 decades of saving digital photos, I question the utility of it.
My parents have photo albums going back a few generations. The family tradition is that they are passed to the cousin with the most children, to maintain for future generations. If I were handed 500K+ images in a drive or a cloud account from each family member, I doubt I would ever look at most of them. With a bound and curated album, I know they are full enjoyed by every family member with an interest in the past.
Awesome! I'm a huge fan of Photos for organizing photos from my external camera but don't always want to mix them up with my iCloud Photos library. I'm gonna give this a try. I wish Apple would allow iCloud Photos to handle multiple libraries like this.
My wife and I now have an annual tradition of making physical albums of memories from the past year.
Making the album in January is not just fun - it makes you mindful of fond memories at a time when you're likely to think "Geez, another year has passed so quickly".
My only regret is realising I should have started the habit much earlier.
I've made photo books but I really try to decrease the flow of paper and other physical things into my house. I'm behind at the moment but I heavily curate/edit and upload to Flickr which also lets me access photos when I'm not at home.
I have PhotoSync (https://www.photosync-app.com/home.html) set up to automatically send all my iCloud photos to my home NAS which automatically backs up to Backblaze. I can also specify other destinations for PhotoSync and the NAS backup.
Just to add context - I do have backups, but they're spread across a myriad of different hard drives. I had used Amazon Photos to consolidate, and benefit from the facial recognition to be able to quickly locate pictures of family members quickly.
I think that I'll give DigiKam another go, against the NAS loaded up.
Not that it helps you now, but i also keep all our family photos in the cloud (iCloud in my case), but at the same time i have a small ARM machine at home that keeps a mirror of the iCloud data.
That ARM machine also has the responsibility of making backups, local to a USB drive, as well as to another cloud. Not mirrors, but proper versioned backups (as in Restic, Borg, Arq, Duplicacy, Kopia, etc).
I also maintain a couple of USB drives with yearly updated mirrors of the entire photo library. The drives are stored at geographically different locations, and surface scanned, updated and rotated yearly.
And finally, as a "last ditch recovery", i maintain an archive of M-disc Blu-Ray discs that contain a complete copy of our family photo library. Every year i make an identical set of discs containing the past years photos, and these sets are stored alongside the USB drives.
I don't bother archiving documents as everything that is important is stored on government servers anyway, or exists in hardcopy. Also, if every step in my normal 3-2-1 backup scheme has failed and i need to recover from the archive, i probably have bigger issues than retrieving my budget for this years finances.
As a fellow small ARM machine owner, what's your strategy for getting the photos from iCloud? Is there a tool one shouldn't feel weird to give their iCloud credentials to?
I was doing just this for a while but stopped because my Mac Mini was too old for the last few OS releases. I then switched to using the Windows iCloud client but a bug from ~2 years ago that consumes tremendous CPU cycles made that less than ideal. (The best you can do is lock it to a single thread, which will then use 100% 24/7)
Now I just don't backup my iCloud, though I do remove everything older than one year every new years to my home server which follows a good 3-2-1 backup strategy.
TL;DR: If you go this route, try to get a Mac Mini that can run a supported macOS for some time to come.
I've been using Windows iCloud client to backup to a VPS. It works okay for files, but for photos it pegged disk usage even when there weren't any new photos to download and my VPS provider wasn't happy. So far my solution is Windows iCloud client for files, and then OneDrive on my iPhone for photo backup, with OneDrive again on the VPS.
I like how the backup is outside my house, but I'm about to add Yubikey to my iCloud account and I'm not sure the Windows iCloud client is going to like that.
I was actually running ESXi on my Mini for a while and successfully installed macOS in a VM on it. The performance was horrendous though, so much of macOS depends on GPU acceleration which I didn't get. I think I've read newer macOS builds don't even have a software video fallback, though that might just be the Apple Silicon builds which wouldn't apply to me.
It was definitely a fun project even if not terribly useful.
It doesn’t talk about not being able to access person tags. Not being able to programmatically access the data about who google thinks is in each of my photos has been an annoying pain point for me for years. Last I checked, the data is also not included in Google Takeout dumps.
i have been using icloudpd for years. there is a docker image that makes it simple to install on my synology. because of 2fa, i have to re-authenticate it from the terminal every couple of months, but that takes one minute.
I tried this but I think my issue is the library is too large. It can pull a few dozen or a hundred or so photos just fine, but then it will time out but not quit out, so I have to babysit the process. Maybe there's a flag I missed in the documentation to retry downloads, but it basically was a nonstarter to me in the state it was in. I think its apples fault though; I can't get a big zip file of icloud data to succesfully download with their website either, it will also time out. Likewise when I try and update the bootcamp drivers on my intel bootcamp machine, I will get 30% of the way there and then it times out. I'd blame my home connection, but from cursory glances at various forums this is apparently a widespread issue with these sort of downloads form Apple.
I have recently switched to a M1 Mac Mini, and just have each family member sign in to that using Remote Desktop. It brings the added bonus of working as a content cache for anything iCloud.
My only gripe is that it downloads the shared photo album (new in iOS 16) once for each account, and when your photo library is 1.8TB, that suddenly becomes a lot of wasted space. When it comes to backing it up the backup software deduplicates the data, but not for the initial storage.
I really wish Apple would implement some kind of method for backing up photos stored in the cloud without the need for mirroring them.
Before the M1 I was using iCloud photo downloader ( https://github.com/icloud-photos-downloader/icloud_photos_do... ) on a Raspberry Pi 4 which also worked well, but in the end I got tired of iCloud credentials expiring every ~90 days, requiring each family member to login again through a console.
Considering the M1 idles at roughly 20% more than a RPi4 (M1 at 4.5W) it was an easy sell. I just got the cheapest model and added a large USB drive. Using a Mac also gives you the possibility of using something like Backblaze Personal with unlimited backup storage, if that’s your thing :-)
I use Healthchecks.IO ( https://healthchecks.io/ ) to keep an “eye” on the backup status (and other more mundane tasks like monitoring the power state of my summerhouse)
You have to fuck around with the storage format because by default scanning sofware, even when scanning documents, will basically just take poorly compressed high resolution images. Maybe there's some service which sorts that out for you but so far I have a little webapp which stores an index of paper copies. Once I figure out how to automate scanning, compressing and OCRing documents then I think it will be worth storing digital copies. But for now it works.
I do scan some paper but there's increasingly less of it and I'm likely to need future access to so little of it that it's mostly not worth the trouble.
I still do a 3-2-1 backup of documents with 2 versioned backups, one at home and one in another cloud provider, just like with photos.
The archive however is the recovery if I’m not able to retrieve my normal cloud copy (hacked, ransomware, loss of credentials, etc), I cannot access my local mirror copy (ransomware, dead disk, etc), I cannot access my local backup (dead disk, separate from the mirror disk), and I cannot access my cloud backup either.
For all of those things to go wrong a the same time, something major has to happen. Besides, where I live, most required documents (drivers license, passport, birth certificate, tax records, etc) exists in government databases, so all I have at home will be various documents that maybe have sentimental value, but not exactly needed.
Furthermore, documents change “frequently” where photos tend to be somewhat more static, so I can archive photos, and maybe get <10% “duplicates” due to later edits, archiving documents will pretty much be a lot of duplicates each year.
That being said, I think we have like 1GB documents in total, so it would be easy to fit in the archive.
I do occasionally want to retrieve older documents. As you say, they're smaller and the effort to put individual docs into a "I might want this someday" folder seems more trouble than it's worth. Of course, having a vast sea of basically write-only docs does make finding things harder especially as I don't put as much effort into organizing things any longer.
> I think that I'll give DigiKam another go, against the NAS loaded up.
I'm also looking for something FOSS that can do basic face recognition + maybe even more, but last time I checked DigiKam's detection didn't work so well, or maybe I got spoiled by the detection in Google Photos. If you do give it a go, would be nice if you reported back on your experience :)
Not sure if it has been mentioned already but photoprism added face recognition a few releases ago and it is working well for me: https://github.com/photoprism/photoprism
The quality of this software is overall extremely good. It is a solo-developer as well so you might want to consider sponsoring them if you end up using it.
Seconding photoprism. The face recognition was pretty meh when it was first introduced, more of a gimmick than anything. But (I assume) a recent update pushed it into "wow, this is actually really good" territory.
Thirding photoprism, got it hosted on a raspberry pi and made it accessible outside of my home via Tailscale. Seems to work fairly well... though I could probably make it faster with an SSD vs an HDD over USB.
To photoprism users: do you know how well it handles identifying photos of kids over the span of years?
Google photos absolutely blows me away with its ~1% false positive rate (that is, given a photo, correctly identifying with kid(s) are in it) identifying each of my kids, nieces, and nephews. I don’t really know what its false negative rate (a false negative would be an uploaded photo of John but that doesn’t get tagged as having John in it) is, though.
I can search for each of my kids and see photos of them going back to their birth.
Photoprism is absolutely amazing. It was the quickest donation I’ve ever made to an OSS project.
I have a Photoprism instance running on my home server backed by a raidz1 ZFS pool (1st backup). Photos are periodically synced to a Backblaze bucket with versioning enabled (2nd backup). The source of most photos is an iOS device with iCloud enabled (main copy). I rely on PhotoSync to periodically sync from the iOS device to Photoprism.
What I found in my previous search is that either they're really good at managing large collections of photos, or they're really good at doing various AI/recognition, but none of what I've found have been good at both.
Hence my interest in DigiKam as their management features for large collections is second to none.
Isn't that OK? You should be able to find some reasonable workflow around using the recognizer to tag the photo in its EXIF/IPTC metadata, and then load that into the organizer.
It's not not OK :) Just not what I'm looking for, I spend enough time professionally with writing glue code, that I'm looking for a good out-of-the-box experience for something like this.
There's a difference between face detection and face recognition. Last I checked (less than a year ago) it could find faces, but was abysmal at recognizing them. I spend a few days on it, tweaking this and that, taking to the devs. It was crap.
I would be careful with hard drives. As I learned couple of days ago when I got my 10 year old hard drive from storage, I was unable to read most of the data as the drive would start to intermittently disconnect from the computer.
Most likely the drive is okay, but the board has developed an issue - maybe capacitors gone bad or something.
Now, I am yet to check if it is the one that has hardware encryption (I supposed to get rid of all of them and copy data to alternative drives) - there are drives that encrypt the data by default and if you don't set up the key, the encryption still takes place with some default key. If the board dies for some reason, then you won't ever be able to decrypt the data - even if technically was unencrypted, even by swapping the board from a working hard drive. This happened to me once and I lost 3TB of important data couple of years ago.
That being said - I think the best additional backup is to store important things on BluRay - producers estimate they should last 80-100 years and so far never had a bad disk, even those burned years ago.
Latest release failed the tests (with some memory corruption yikes) even. Doesn't inspire a lot of confidence that it won't eat your photos.
But the v7 uses DNN from OpenCV for face detection that supposedly has 91% accuracy on CPLFW which is pretty good! So how come people here say it sucks?
I'm away from my desktop computer, but it wasn't half bad when I last updated its DB. I don't take much portraits, so I don't use that feature much, but give it a look when I can and update you.
I'm glad you haven't really 'lost' the photos since you have copies elsewhere. It is surprising how many people these days rely on some cloud service to store and preserve their only copy of some important and unreproducible data.
You are right about file systems doing a lousy job of helping you organize and locate photos, especially when they are spread across many different HDD and SDD drives attached to various computers. They are equally bad at organizing other forms of content (documents, videos, logs, music, software, etc.)
File systems were invented decades ago when the biggest hard drives could only hold a few thousand files at most, and drives were also so expensive that most people only had one of them. Traditional file systems are antiquated and need to be replaced with something better.
Self-host and use Photoview [1], it has face recognition. I use this since 2 years with 5TB of photos (on ZFS) and it works flawlessly. It is worth it - everything under your control and you decide when to migrate. My phone's photos are also uploaded automatically to a Nextcloud folder, which is read-mounted to Photoview and scanned automatically.
* A nextcloud+recognize (for face recog) instance running on Hetzner. $20/month (I co-host other services on the VM). This is attached to a Hetzner storage box.
* Syncthing pushes all of nextcloud to a zfs.rent (great service BTW) machine, I purchased my disks up front, so it's $cheap/month. ZFS snapshots are taken.
* Local RPi NAS. Syncthing up to the other two.
About $40/month in total. How valuable are 240k photos to you?
Uploading photos is a matter of sending them to Nextcloud. Syncthing does the rest.
I'm curious if you've asked your family members whether they're ok with having photos of themselves being used to train Amazon's facial recognition model? I know it's a common practice and Amazon isn't alone, yet I feel people should be more aware of what's happening.
Digikam is less crashy than it used to be. Its slowly going through my 650,000+ photos and finding faces. I started this past Sunday. I expect to be done in a week or two.
It literally only has been 1 hour since OP posted that thread.
I agree with all the points about having backups and/or using better service/self host etc., but IMO it's too early to conclude anything about this incident.
I'd be freaking the fuck out too, tbh. These days you tend to get better support from the megacorps if you tweet or post on HN, it can bypass the obligatory outsourced support layer.
The stories that get helped often start with the premise that a company has destroyed the person's life/business and the company is hated by a large portion of people here.
That will get the most upvotes, visibility, and sometimes a company will reach out.
Yeah I usually have bad experiences calling Amazon and have stopped buying anything from them, preferring wither other online stores or driving to a physical store depending. The CSRs usually don’t speak english well enough to respond to anything other than the issues the app also lists. One CSR insisted I send back an item that fell out of the package in the delivery driver’s car.
I have too, but when they have a bad policy they’ll defend it to death.
Specifically I contacted them about a product listing that, at best, was word salad right out of Blueler’s book on schizophrenia. They told me they didn’t want to hear any complaints unless I’d bought the product.
Of course, that kind of listing is endemic on AMZN, and I’d have no problem reporting several of those each time I go shopping.
Actions have consequences though and AMZN is now the last place I look when shopping for things since shopping there is somewhere between visiting Mariupol and the South Bronx in the 1970s. It is a matter of time before I pay my last recurring bill of any kind to them.
> They told me they didn’t want to hear any complaints unless I’d bought the product.
You have to think about the game-theoretic equilibrium of the alternative, given the same set of bad actors involved: if you could report listings you hadn't bought, then these idiots would all just be constantly reporting their product-category brand-name competitors' listings, as non-suspicious one-off actions from different dark-web-purchased pre-aged accounts, to the point that Amazon would just have to ignore "reported listings" as a signal anyway.
I never see senseless product listings on Ebay. For that matter, I get Ebay shipments from Japan faster than AMZN gets me packages from a warehouse in the next state. People selling animals and farm stuff on craiglist sometimes struggle to spell words like ‘cow’ and ‘pig’ but once more the listings make sense.
Do you believe that this is because Ebay knows something that Amazon doesn't know? I personally think Ebay just doesn't drive the kind of sales volume — or have the particular expected sale+logistics lifecycle (immediate payment to enable immediate shipment of locally-warehoused goods through Prime, etc) — that makes these particular scammers interested in exploiting it.
Also, as you're describing this more, I'm increasingly confused, because I don't think I've ever seen an Amazon store listing that is truly "senseless" in the way you're describing. Maybe it's just that I've only ever visited Amazon's Canadian storefront, and these particular attackers aren't active on there; but while I see lots of AliExpress-dropshipped product listings, lots of page grooming, etc., but never a completely-senseless listing. I believe you that they exist; but can you link an example?
In that case, you're wrong: the Denial of Service is "jamming the human review queue", and it would be very successful. Even Amazon doesn't have infinite money to hire moderators to keep up with the number of [99.99999% automated and illegitimate] reports that would flood in.
(Actually, it's probably a supply problem more than a demand problem — there probably aren't enough people willing and able to work as moderators on the entire Earth, to manually review the number of complaints that would be generated here. It would cost them orders of magnitude less to just hire people to manually review all changes made to product listings! ...which they still won't do, because that's still apparently too high a cost to bear.)
The point of these services is they're supposed to be an easy-to-use, worry-free photo backup. Amazon/Google/Apple should surely have the resources available to keep my photos stored somewhere with some redundancy. I don't want to worry about how they do it, but that's the idea. In exchange they're probably training image recognition algorithms or what have you on my photos, and I certainly wouldn't keep anything too personal on those services (i.e. nudes). I may even be paying them for more than the free storage limit. The least they can do is be a reliable backup. Yahoo Mail doesn't disappear emails randomly, YouTube doesn't just disappear videos randomly (barring stupid robot moderation). You'd expect the same for your photos on these services.
When I look through my "Watch Later" list, probably 10% of videos are missing. I think you have to choose "show deleted".
I've been planning to set up a script that uses yt-dlp to auto-download the new videos on my Watch Later list every night, but haven't got around to it yet.
At least for code, git being distributed means that there's always multiple copies with typical workflows.
For example, in addition to the copy on GitHub whoever pushed last should have a completely current copy. And everyone else working with the repo should have something pretty current.
In macro photography, taking an extreme closeup of something like an insect necessarily has a very, very thin depth of field due to its high magnification. To get the entire insect in focus, you use a technique called focus stacking where the focus changes by tiny fractions of a millimeter on each exposure. You can literally take 100 to 200 of pictures for one subject. If I do 10 of these in a day, I've reached one percent of OPs forever photo volume.
https://www.photography-raw.com/creating-stunning-macro-phot...
Time lapse photography is taking a photo every X seconds/minutes/days. If you want the video to look smooth, you need 24-30 shots per second, so a 30 second video has 720 exposures. So again, failed three time lapse attempts and I'm at 1% of OPs forever exposure count.
There's an awful lot of judgment in your comment, and the issue certainly seems like a lack of understanding on your part, rather than OP's "psychological issue" or "lying." I'll refrain from my own judgements, and suggest you watch David Foster Wallace's "This is Water," and when you're done, please watch it again.
https://www.youtube.com/watch?v=eC7xzavzEKY
Knowing Amazon retail / services support, their data is gone and they'll be lucky if they raise anyone at all.
If you pay $15k a month for AWS enterprise support, you will find someone who may or may not be able to help you within 6 months after misinterpreting the problem and having to raise your account manager and wave your hands around.
I'm surprised, Amazon is well known of having the best in class customer support for their store (I mean: "Customer: My kindle broke, because I sat on it. Amazon: no worries, let me send you a replacement, no need to send back the old kindle" level of good support) - oh and human support, not google botocracy.
Any large-scale consumer-grade service optimizes for cheapness of support.
It's just a coincidence that when it comes to retail, the cheap option (just ship another one out) also works well for the customer. At scale, shipping duplicate items, even expensive ones still works out cheaper than well-trained, competent, well-treated and well-paid humans.
This one on the other hand requires engineering time (can't be resolved by a bot or monkey pressing a "ship another item" button) which isn't cheap and unless they have legal liability, it's cheaper to just stonewall or even ban this customer forever and lose their business.
Have you tried getting through to a human recently on Amazon retail support? It's quite difficult. First you have to negotiate with a bot with an IQ and service level of an angry wasp, then you get filtered through to someone who usually can't help. So you have to try 2-3 times to get stuff sorted.
The whole thing is designed to put people off and make it difficult to get problems resolved.
I'm fairly good at being annoyed enough to work around this but most people out there can't handle things like that easily and Amazon knows it.
Last November, my Logitech mouse's scroll wheel started acting funny. Tried to check on Amazon the warranty situation, the docs said: take it up with the manufacturer (note: the mouse was sold by Amazon, wasn't a marketplace item). This was like two weeks before the 2-year warranty was up.
So, I headed on over to Logitech. Man, their site sucks. My mouse was a "gaming" model, so it wasn't directly clear where I should seek help. I end up with some support guy who was obviously doing something else in parallel, judging by the time between replies. Went through the usual checklist (is it on / did you try unplugging and plugging it back again?). Never mind that before getting on the chat, I had to fill up a form with the exact same questions, up to and including the serial number, which is the first thing the chat person asked for. Half an hour later, he asked the killer question: where did you buy it? Amazon. Oh, take it up with them, then.
Went back to Amazon, opened a new "other" case. Less than two minutes later, I was on the phone with someone from Amazon, asking me point-blank whether I wanted them to try to fix it or reimburse me. Said the latter, and he asked me to be sure to send it within 30 days.
Guess where I bought my next Logitech mouse?
So yeah, Amazon's retail support experience is still top-notch in my book, at least on this side of the pond. This was for an account without prime support that doesn't do a lot of business with them.
I'm fairly confident that if I actually opened the support request with Amazon first, the ending would have been the same. At no point did Logitech say that their online support was only for items bought through their online store.
Sure, Amazon could have said "yeah, call us" instead of saying that I should see the manufacturer, and it would have saved me half an hour.
But I was quite disappointed with Logitech's reaction.
I've had an issue with a bag and when I took it up to one of their stores for repairs, they just took it in for repairs for free without any question asked whatsoever. Not even where or when I'd bought the item (I didn't buy it from their name-stores, but from some generic department store). Didn't ask to see any receipt. Just "It'll take around a week. Can you leave it now?"
Note that this wasn't some fancy boutique bag or brand, it was barely pricier than the mouse.
Since I'm complaining about Logitech, I should name the brand with great service: Le Tanneur, a French leather goods company.
It depends on "recently".
I contacted them (human, right away, speaking my language - I'm in EU and the store was in different country than mine) about a year or two ago where I had issue with my phone, and there was no issue to get my money back (Pixel 4 had issue with NFC not working correctly) - no "We will repair it", like any other retail store I used.
Last month here in UK. After a package was stolen from my doorstep but the delivery driver said it was handed to receptionist. We don't have a receptionist, or a reception!
I'm a "high hitter" customer. I have prime and spend a hell of a lot of money with amazon every year so they tend to do what you ask eventually. But if you're a new customer or low value one your customer experience will be very very different. I know several people buying 1-5 things a year who have been told to piss off.
"The 'computer' is lying" situation is a problem generally. I didn't receive a 16-port switch and another small item a few years back even though it was noted as delivered. Neither Amazon nor UPS would do anything about it. A couple weeks later (I think it had been snowing) I discovered the package tossed in a ditch to the side of my driveway. Everything was actually fine but I had reordered and didn't have a need for a second switch.
I've had a similar situation, where the delivery guy said I wasn't home, so he left the package at some drop-off point. It was a small package and the house has a big ass mailbox. There wasn't a drop-off notice, either. And both my parents and I were at home, with my mom hanging around in the garden in plain view of the mailbox.
The opening hours of the place were a PITA, so after two attempts I called up Amazon and complained about this. Sure enough, they sent the package again without trying anything funny.
This has not been my experience. I pay 100$/mo for support and when I have had issues (had a RDS upgrade go really bad once) they have contacted me quickly to get details. Usually only 2-4 hours for an engineer to be on the issue.
This space is frustrating with any of the cloud vendors as they originally offered a lot of storage for free, then had to resort to various strategies to claw that back after it got expensive.
Google is especially frustrating for me. By default, an Android tries to sync all videos and photos up to Google photos. You can turn that off, but not in any sort of granular way. For example, you can't say "sync photos, but don't sync videos". You USED to be able to do this, but it's all or nothing now. Which I suppose drives revenue for the paid storage plans?
While I still use Google Photos, there are times that I don't want pictures I take with my phone to be synced to the cloud. Since Google Photos does not allow such functionality, I installed a second camera app. That app allows me to choose where the picture are saved, so I save them to a folder that doesn't get backed up by Google Photos.
It's annoying to have to do this, but at least I can finally choose what gets synced and what doesn't.
I specifically use "Secure Cam" from GrapheneOS (mine came from Google Play) for this, as it is close to the stock pixel camera and doesn't suffer the lag some other camera apps have.
-- edit
I originally called the app "private cam" and said it was from fdroid, apologies! Forgot I had aliased it.
> This space is frustrating with any of the cloud vendors as they originally offered a lot of storage for free, then had to resort to various strategies to claw that back after it got expensive.
Getting users to add photos and then charging (while also using them as image sets for training) was the strategy from the beginning. They didn't resort to anything. Keeping your photos stored somewhere is very sticky, particularly for non-software developers. And even so how do you even transfer them to another service without a huge bill?
I did spell out what one of the changes was. I used to be able to sync photos, which for me isn't a lot of space...but also NOT SYNC videos, which do use a lot of space. That option was removed, and syncing can only be turned off/on globally now.
This is just a reminder to take a backup of your cloud photos and store it locally (or other cloud provider). For example, with Google Takeout you can get a ZIP file with all your photos and videos from Google Photos.
I hate how Google Photos seems to no longer have the ability to auto sync to your computer. When it was integrated more with Google Drive you could do that, and then my daily automated backups would pull them. That way I had them in Google Photos, my computer's hard drive, and my off-site backups, all automatically. Now my only option is to periodically do a manual Takeout, as far as I can tell.
So... what do y'all actually do with your hundreds of thousands of photos? I just bulk delete everything older than a month when my phone is low on storage. Do you ever actually go back and look at any of the hundreds of thousands of pictures? When do you do this?
When you have kids, and when they turn into teenagers, you'll want to remember them as they were before.
;)
My daughter loves looking back at home videos of the family. So embarrassing, so beautiful, so many little things about their personalities that would otherwise be forgotten.
Just a few hours ago I was checking backups and stumbled into a nest of folders that had some more home videos in it, and of my son just being his quirky little kid self. Gives my life meaning, shows me I did stuff ok.
I have photos backed up on Google Photos for the last 10 years. I go back to find old photos all the time. There's different reasons, sometimes I'm just feeling nostalgic for a time in my life and want to see photos for them. Sometimes I want to find a specific photo of someone to send to them.
Here's a practical example of it coming in handy: back in university I moved to another city for an internship and took some framed photos I had on my walls. When I moved back into my parents' house when the internship was over, the nails were still in the walls and I wanted to hang them the same way I had before but couldn't remember which photo went where. So I went into my Google Photos, typed "photo frames," and was able to find pictures in my library with the framed photos in the background, and piece together which photo went where to get everything back to where it had been before.
All the time actually. I have monthly folders, automatically backed up by my phone to the NAS. Each month my wife and I sift through the photos and delete all duplicates and boring stuff and then we have around 150 photos per month that we sometime just have on as a slideshow.
Yearly, we create printed photo albums that we look at regularly and show to our kids.
But all this only works after lots and lots of hours of manual work filtering the photos.
Still, over the last 15 years or so for two people we have way more than 100k photos stored on the NAS.
I look at them a good bit. Having a good tool like Google Photos or similar makes it easy to jump back to the photos from that one trip a decade ago, that holiday together with the family no longer with us, the dreams I had for myself in highschool. I find it interesting to see the changes in my life objectively by looking at myself, the places, the people, and things around me at various points in my life. I find it helps me continue to keep perspective of my life in the same way some might keep a journal. It can help me jump back into the shoes of who I was 5 years ago, 10 years ago, 20 years ago, and reflect on who I am today and where I'm going.
If I don't have any real reference points, its hard to measure things. Its hard to improve things you cannot measure. And sometimes it just feels good to help remember the good times with people no longer with us.
The thing is also to delete photos you don't want to keep long term. Most of my photos are only synced when I am on my local wifi, so until then having a good hygiene of deleting badly taken photos right after taking them.
Having said that I have recently started (late december) to do a weekly printing session of my key moments of each weeks. Well there have been weeks in january and february I didn't print anything. I sometimes only have one or two photo per day/event but since I put the date on the notebook next to the photo and keep all other photos it helps me find back the whole event if I want to see them all. I am using smaller photo format than 10x15cm, mostly zink 2"x3" adhesive photos and kodak instant print 3"x3" from instant cameras that have a printer function.
I've been enjoying the iOS photos widget. Every day it surfaces new photos from the past, based on different themes. A nice 2 minute diversion each morning.
But you're right, otherwise I would never have time to dig back into old photos.
Fun party trick: look up old photos of people when you meet them after a long time and show it to them. Everyone is shocked/embarrassed. Extra points if you can pull up childhood photos of them.
I liked that the X-Files reboot kept the old credits. Mulder and Scully look like babies. Then the credits end and reality intrudes. Of course, I'd aged at the same rate (maybe faster) but it was still a shock.
I have no idea why you're getting downvoted. The icloud-photos-downloader is an amazing tool. I run it from a monthly cronjob to have a local backup in case icloud iexplodes.
Local photo library is a folder, disguised as a package. One of the folders store all the originals on the device. Copy that folder, and you're good to go. You can also get the full folder hierarchy, if you wish.
If you enable iCloud for the photo library, and even if you select store originals, I have found that Photos still offloads some pictures. It’s just not guaranteed.
I really wish Apple had a complete export button, similar to google takeout. Or at least a good CLI tool to get the original images one by one.
I rsync images out of Apple’s weird file structure that’s in the bundle. It’s horrible, but with ‘download originals’ enabled it works well. It’s particularly frustrating with large libraries as they occupy nearly all an SSD. For this reason I do it all on a VM that’s on a Synology.
I had a pretty big incident mid last year, which will turn into a dramatic blog post eventually.
I didn't lose anything I've yet noticed. Very happy with how well the backup strategy held up, although potentially quite lucky the second on-site backup was unaffected. Didn't have to dip into the complexities of the off-site chaos (which reminds me...).
One of the nice things about having an off-site backup is that, in addition to being offsite, it's presumably a completely different backup process so if the local backup wasn't working for some reason (and you didn't notice) you're probably still covered.
Having a spare hard drive that you occasionally sync up and drop off at a family member's house in addition to other backup strategies is indeed a decent idea.
However, I'm guessing that--if you're anything like me--you get lazy about refreshing it and, if something happens, you realize it's been a year since you did a fresh backup.
A cloud provider backup (as opposed to sync) is a good belt-and-suspenders cheap insurance backstop to local backups that you hopefully never need.
I have local Time Machine and Synology NAS but I also pay Backblaze a nominal amount. Companies were paying Iron Mountain large sums of money before there was a cloud.
You can also say: cloud servers are on fire and the house is on fire, have another backup someplace else. It's all probability.
The probability that your house will be on fire these days is low, and the probability that your computer/hard drive will get on fire is also low (since you will probably take it out, no one will leave his phone in a burning house right?).
We can say that the probability above is that low that it doesn't really affect anything. There are many other different parameters to add, and I use a bit of math here which doesn't always reflect on life, but you get the point.
This is a general question; is no one else worried about Amazon or whichever company trolling your photos with face recognition, building up a graph of who you are connected to, all in the name of "giving you a better shopping experience"? Or who knows what else they might do with the meta data or AI object recognition shenanigans.
I guess I don't extend much trust to the companies that provide these services, especially if they are free. But then I don't take many photos nor share them around much. That might change the equation a bit if I was more into that. What photos I do take that I want to keep I download to my desktop.
I don't trust to open my network to the internet though. So I've created a VPN to connect with my ubiquity network from mobile. Sadly, I can't share photos with other folks or allow my family to participate easily.
I'm a layman in terms of networking. I always wonder how people can sleep if they open their home network. In this case either by using QuickConnect or forward ports and access the machine through a custom domain secured by TLS. I mean that would be perfect but I just can't convince me to take that burden of maintaining that open door into my network.
I've read about creating sub networks with the power of ubiquiti (or any other router), but I will never trust anything I configure there either because I can't confirm it's secure.
So I'm stuck with Google Photos additionally to Synology Photos because of the convenience of sharing.
My solution is to not open it. Use the cloud for browsing, use the home machine to copy the files down. You rarely need to browse on the home machine, it's really just there if the cloud provider ditches you.
I own a Synology NAS, and now believe that it would have been easier to just set up a computer for the tasks I use it for. There are just too many cases where the solution is to use a container (and the clumsy web interfaces for reverse proxying &c), and if I end up SSHing into it too much, I prefer to have a more controllable environment.
It does fit some workflows very well though, you just have to assess if your needs over the next few years fit their business intentions.
Synology has an app which will automatically back up your photos to the NAS/server (only when on WiFi). I haven't been able to find an alternative to this on Android, have you? (photosync looks good but seems to only work for iOS last I checked).
+1'ing this too. Very user friendly with its web interface. And I think it is the only self-hosted solution that can backup live photos from iOS devices.
There is only one significant issue that I found: Backing up large videos fails If I put the Syno behind Cloudflare. Because the photo backup app uploads it as one large HTTP form data and CF does not like it. So now I need to make do with basic firewalling at my place.
One of my worries that keeps me sleeplessly awake at nights sometimes is the possibility of solar flares wiping out storage/data -- at my bank, on my hard drive and what not. A sufficiently strong solar flare can essentially put us back a few centuries electronically, and a few millenia socially, instantly.
Wonder if there are stress tests on cloud storage that account for this possibility by means of adequate insulation or whatever it is that can counteract a solar flare.
> possibility of solar flares wiping out storage/data -- at my bank, on my hard drive and what not.
Oh, one solar flare can solve global inequality in an instant! There money in all bank account would dissapear and the recently rich would have to pull themselves up by the bootstraps!
Blockchain is much more resilient than regular databases. There are thousands of copies of it across the globe, some are stored deep underground, and even some were launched into the space, I believe.
I keep my data in the cloud, with a local 1:1 mirror of which i keep a 3-2-1 backup (local + cloud).
Besides that i keep USB drives and M-disc archives of photos, identical sets stored in geographically separate locations. Optical media should be resilient to most natural phenomenon like flooding and solar flares. They're not resilient to fires, which is why i keep duplicate sets.
I keep everything in the cloud as well. Documents are either in GDrive / Dropbox and photos are in iCloud. For sensitive documents I created an encrypted disk image that I store in Dropbox and mount when I need access to it, which is pretty infrequent. I periodically backup content to a NAS and I backup the NAS to B2.
I use Insync since this seems to be one of the few softwares that works on both Windows as well as a Linux box. But it has crazy privacy concerns. I have been able to view documents uploaded on someone else's account on my Insync app! I have no idea how many folks have seen my documents/data.
Take a look at Cryptomator. If offers transparent encryption across windows, Linux and MacOS. Desktop clients are free, iOS/android costs a one time fee.
"A lot"? Sure, I guess. But as someone who stores multiple copies of everything both onsite and in the "cloud," I certainly don't have local tape backup and I doubt that Backblaze does either.
When I hear individuals on talk about their backup routines they have developed, home nas servers, scripts that keep things synched, etc, I can’t help but to think about failed crypto ux attempts.
I don’t like relying on monolithic monopolized corporate services as much as the next hacker— but I think there should be an easy to understand service for the 98% of basic users.
If someone could make a service that detects and requests permission to connects to all of your cloud services and provides full backup service that includes whatever the best practices are it would seem like a win for those who don’t want to roll their own.
A marketable “sticker” Ala “intel inside” ie - “open backup compatible” would be good to have with a simple but serviceable api.
I think today there is little financial incentive for storage providers to make the data easy to access via machine.
Ideally an api would not only allow incoming data to be quickly compared and synched… but would allow graceful and intelligent transfers or splits or duplication levels among stores, in the case that you want to change services or have your own backup.
The problem is no one wants to learn how to do it. Everyone wants the easy button. This is why we have the cloud, iCloud storage, google photos, amazon photos, back blaze, etc....
None of this is that hard to do, not one just wants to put a minimal amount of effort into learning how to do backups, or automate them or deal with DR scenarios. If this information is important to you it would behoove you to learn how to protect it. Otherwise its not that important to you. Obviously IMHO.
It doesn't help that the OS vendors have been letting their own local backup tooling rot over the years, probably in favor of selling more cloud subscriptions. When is the last time Time Machine got some attention, when it was released 15 years ago?
As a slight aside - I was surprised to learn that Amazon Photos (if you have Prime) includes RAW camera formats as valid photos for the purposes of their unlimited storage offer.
I've taken to essentially backing up all my RAWs to Amazon Photos - even the ones I would typically discard.
I also. I don't know why they would offer that as part of their base Prime service when no competitors seem to. The upsell for printing doesn't seem appealing to the type of person who would be storing RAWs.
This is why you should always synchronize you cloud with your other cloud, and the NAS in your closet, and floss after every meal, and take daily vitamins.
From what I can find, Amazon Drive is shutting down at the end of 2023 but Photos is not. So you've got until end of year to download non-photo/non-video files from your Drive before they're gone.
Lots of reminders to make backups and alternatives offered, but I think it's fair to expect something from Amazon here since Photos, being a Prime benefit, is a paid service, which should come with a reasonable expectation of stability. It's a shame Amazon doesn't have an easier way to find formal support.
> Local backups shall be stored in different locations (depends on importance) not reachable by network. The cloud is the server of someone else and not a backup.
That's common knowledge among tech people but blaming the victim is not the solution here. These services are marketed as safe, reliable, and not needing backups (in fact sometimes they make it difficult to take backups, even more so for a non-technical user).
Maybe the regulation should be changed that any consumer-grade storage service either needs to provide adequate SLAs & compensation, or have to advertise in bold and prominent font (at least as prominent as their selling points) "WE MAY EAT YOUR DATA AT ANY TIME, PLAN ACCORDINGLY".
Nobody is blaming the victim, they're saying that you should have a backup outside of your cloud provider.
I don't think people care enough about SLAs to pay a premium, until something happens to their data. But by then it's too late. The bold warning is cute, but have you seen cigarette packaging in Europe lately?
> The bold warning is cute, but have you seen cigarette packaging in Europe lately?
The warning served its purpose quite well though - the majority of people including smokers concede that smoking is harmful. If they are still smoking despite that, it's on them.
The same cannot be said of cloud services. Very few people outside of tech think that the cloud can fail (whether technical failure or just randomly ban you for potential unspecified violation of ToS), so the same kinds of warnings would be useful to spread awareness.
> If they are still smoking despite that, it's on them.
That's my point: the warnings don't lead to reasonable decisions.
"If you don't pay for it, you are the product" has to be one of the most echoed sentiments of the last few years, and people still demand everything for free, and prefer to pay with their data instead of paying for the content they consume (e.g. on newspapers).
If Amazon gave them a choice of "you're data is most most most likely fine" and "your data is most most most most likely fine, and we'll pay you $1000 if it's not but you pay twice the money each month", I doubt they'll see a lot of people switch over. They'll have some, but not many, because ultimately, people don't care enough.
> the warnings don't lead to reasonable decisions.
The warnings lead to informed decisions. They may not always be reasonable, but at least the person knows what they're getting themselves into.
My intention is not to outlaw "best effort" cloud storage or legally compel the providers to offer an SLA, but merely to make the limitations clear so that customers, including non-technical ones, can make an informed decision.
> I don't think people care enough about SLAs to pay a premium
Huh? People specifically pay for cloud to not lose their data. Before that my parents used to copy photos from laptop to laptop, and lose them when a hard disk failed.
Now if cloud loses your data, they might ask wtf am I paying for.
There's "cloud" and there's "guaranteed cloud with SLA and compensation if it fails", and the latter will be more expensive. The former is somewhat "best effort", and that's also why it's cheap.
Is it really cheap? For 3 years of cloud subscribtion, I can buy 2 Nas drives, drop them at the homes of two of my friends, sync my PC to NAS 1, and backup NAS 1 to NAS 2.
They are like 2 -click setup if you use Synology/Qnap/etc.
You can do that, the average person can't, so you should probably consider the time you'd spend on it, and how much it would cost the average person to hire someone to set it up for them. I don't think it'll be too cheap.
In the end, not dealing with that is most likely worth it for most people (including most people who have the necessary skill set).
I am not proposing rolling my oen linux box, I am talking about an off-the-shelf product where you plug it in and set a password. You follow a manual with pictures and there is real customer support.
This is very user friendly and good 50% of working-age population can do that.
Man, if I smoked, I'd buy a cigarette case and transfer those bad boys over. I don't wanna look at gross lungs every time I go for a cig. Of course that's the point, but yuck.
I‘m sorry if you interpret here a blame. I consider repeating common knowledge actually a necessity!
To keep tech people informed and aware and also non-tech people.
From a scalability standpoint how does one accumulate, and then utilize, a quarter million photos?
I'm a bit older than most, but that would be a photo every other hour since the minute I was born...
Kind of a digital hoarder moment. Which is relevant because my entire archive fits on a flash drive and I have a couple flash drives and its no big deal at that small of a scale. But a quarter mil pics will take some serious storage.
Now I could imagine if you owned five security cameras and stored a snapshot every minute for a month, that's about a fifth of a million photos, but that's all autogenerated and if you lose your complete set you'll have a new set in a month anyway.
A quarter million is still a big number, but I'll admit to my family having 60,000 photos in storage. They're mostly since we started having kids in 2014, which works out to 20 per day. Of course, some days we take zero photos, and other days we take a bunch on burst because it's hard to get a photo of four kids with none of them picking their noses, let alone looking at the camera. When we go out somewhere, I don't even think we take a lot of photos compared to most people; we're mostly in the moment and trying to wrangle the kids, and I'm shooting with a $200 Android phone. So I can imagine a family with older kids, that's more into photography and social media, easily accumulating 4x our number.
Now, we select less than 10% for a private Flickr album that we actually enjoy looking at and sharing with the grandparents, etc. But so far it's been easier and safer to store the remainder instead of deleting them. I've gotten into trouble before with my wife when I deleted a photo that I thought was crummy, but she loved for some reason.
Edited to add: We also photograph all of the kids' artwork as a compromise instead of trying to save all that paper or hang it up somewhere in our house, which would be their preference. Those go into their own albums, which has actually turned out to be pretty neat.
>because it's hard to get a photo of four kids with none of them picking their noses, let alone looking at the camera.
But those nose-picking moments can be so easily utilised...usually when the kids get to 17-20yo and their partner 'Of course wants to see pics of when their loved one was a child'.
I’ve been taking (digital) photos since 1999. I’ve got 350k unique images, but more than a million unique SHAs.
I’ll maybe take a couple a day when out for a walk. 10-50 for an event. 20-30 for a big family gathering. My kids were always a good excuse to pull out the camera. It all adds up.
If you shoot with a big camera, people frequently opt for “JPEG+RAW” which is two image files every time you tap the shutter release. That adds up.
Also, if you use(d) google photos, or any other service that downscaled your originals, and then _also_ directly take a backup of your DCIM directory off of your phone, there’s another source of duplicate images.
They could be a photographer. Often they'll take numerous photos in a very short time span to capture a bunch of movement and angles just so they can pick the best of the bunch later
Why do they keep everything? Idk, /r/datahoarder is a subreddit that exists though so the mindset is out there
IMO... why not? 250k photos at 10MB each is only ~2.5 TB. That easily fits on modern hard drives. You can even buy SSDs that would fit that for under $500. If you are a professional photographer, then you absolutely should keep everything you ever shoot, just in case one photo is ever worth it. Even if you're an amateur though, that's not really that much storage in the grand scheme of things.
It’s a complete pain if you use a laptop as your daily driver. You don’t want the entire drive filled with photos, and using an external drive with Apple Photos is a PITA as the integration with iCloud doesn’t play nice if the drive isn’t reliably attached.
Apple would make a lot of lives easier if they made a tidy way of exporting images from Photos easier to automate. And while they are there, a photo file structure that wasn’t horrible, with photos names something like ‘date-time’.
Ah yea, if you are stuck using an over-priced Mac laptop as your main machine, and cannot upgrade the storage or use an external hard drive or NAS because of buggy Apple software, that would be a decent reason to keep some of your photos in an exported state instead of keeping the whole archive flat and available online at the same time. Even so, you _can_ buy a MacBook Pro with up to an 8TB drive, it just costs 2x what it should.
Though I’d pay a lot more than I did for my laptop, it’s that good. I regularly see other machines get rated and do try them out but the hardware (trackpad in particular) is never anything like as nice.
Yea absolutely. I use a Linux desktop day-to-day, and a 6 year old ThinkPad on the go, but I'm a software engineer, not a creative professional. If I were a photographer by trade, I would certainly have a MacBookPro too.
That super sucks. Stories like this (but about Google Photos) led to me setting up the following backup strategy.
1. First, all photos are on a fat external harddrive connected to a homelab server. When I come home with photos, I dump them onto this (TODO to find a way to automatically copy them into the proper spot off the camera upon the camera being plugged in, or a different spot if the gopro is plugged in)
2. Once a day, these are rsynced to a separate set of harddrives, which serve self hosted photo software I use (similar functionality to google photos. Looking for a replacement that lets me create arbitrary albums of photos, and then create web pages pointing to those arbitrary albums, that's FOSS. Currently evaluating photoview and librephotos)
3. Once a day, all photos are backed up to a backblaze b2 bucket using duplicati. with several previous versions also kept.
4. Once a year, a fucknormous external drive is purchased, all photos dumped onto it, and mailed to my parent's house, where they put it into a firebox.
5. Oh, and all my photos are also uploaded to google photos, because why not
I'm very much a scrub when it comes to sysadmin tasks and was able to set up all of the above with about... I don't know, no more than 100 hours of effort I'm sure, probably less. I recommend at least looking into consumer backup services like the one by backblaze!
I'm hoping that it's just a stale cache or microservice that hit an issue building its view of the metadata or something. They use S3 to store your data (according to this blog[0]) so my fingers are crossed that the data is still there, they just need to get an engineer onto it.
- Synology NAS is accessible on LAN & phones sync photos when at home
- Nightly rsync from internal NAS storage to 1 external (non-raid) hard drive
The nightly rsync does not delete files, so the external hard drive is pretty close to an off-site backup even though it's still attached. The only better thing I've thought of doing is mounting it ro after backups + re-mounting before backups.
At the end of the day we get 3 copies of data: the photo on your phone, the photo on the shared NAS, and the photo on an external hard drive that spends 99% of its life spun down.
IMO the biggest remaining risk to this is a house fire at night while you are at home. You wake up to alarms and smoke and run out the door without your phone. All three copies of your photos will burn at once. This is where an additional off-site would be valuable. That might be a second external hard drive that you store off-site and re-sync every few months, or maybe an encrypted cloud backup using restic.
Stuff like this terrifies me. I'm personally reaching a point where I'm considering investing in an offline NAS with a RAID array and replacing disks every few years to be safe (or based on SMART tests).
Imagine uploading all your photos to a cloud provider, then having 5% of your files disappear. How would you even know your photos are missing if its a small %? At least when your local filesystem is corrupted, you know what you're losing.
I'm currently in a similar pickle with an icloud account. It's not lost, yet, but I can't pull the photos locally at all. The download links supplied on the icloud website time out a few hundred mb into the download. Third party command line tools also time out a few hundred mb in or so. This is a large library, so when I try and open up the Photos app on my mac and sync it there, the app quits immediately. I feel like its a ticking time bomb of a situation here with these data on this icloud account, and I also feel like I'm powerless and out of options to do anything about it, although I will stop using icloud and similar services as primary storage going forward.
I thought I had received an email from Amazon a few months ago telling me that my storage on Amazon Drive was going away. Pretty frustrating any time a service like this just goes poof.
EDIT: I misremembered the details, actually the email says:
> Beginning December 31, 2023, files stored on Amazon Drive will no longer be available to customers. We will continue to support Amazon Photos and ensure customers can safely back up, share, and organize photos and videos with the Amazon Photos service.
Is there a way now to pull all the data. This is scary and I’m right now buying USB for storage, but I don’t know how to bulk retrieve all my photos. I imagine they probably close any ways to export the data to not migrate to other service. All I can think is to do a Selenium program to go through all the entries but I think it will be a bitch to have it resume from a certain point and not miss something
It's not clear from the question what the problem could have been, but I would be REALLY surprised if the app would be responsible for removing the photos in storage.
The app is just a front-end, deleting data (with Amazon photos and even S3) is not easy to do by accident without user input.
Is it just resynching the application (metrics updating) ? Super curious the issue.
I urge everyone to use external disk backup, possibly with prism opensource alternative to cloud photo storage. After Facebook deleted my acc for no reason and google locked my account due to payment system problems- you start to rethink who really own your memories in the cloud and how securely they stored.
That is why always you should synchronize your cloud with a local backup storage, always remain skeptical on cloud-backed storage as you are relying on someone else keeping it up (it can go anytime).
Cloud storage is just as volatile as your physical storage breaking or getting lost.
How fast are Backblaze transfer speeds these days? I wasn't a huge fan of the client, but running something third party like Arq[1] felt like a bad workaround.
restic backing up to B2 works very well for me, and it’s really cheap. The restores I’ve tried so far have been acceptably fast, albeit in the gigabyte rather than terabyte range.
I have 4 backups (and growing) of all my important digital things especially my photos. I have two separate overlapping backups and every few year I buy a new hard drive and put the old backup drive in storage. I will continue to do this until I die.
After a scare of being temporarily unable to recover my account on Flickr (later resolved), I got into the habit of printing my significant photos in a shop and storing them in a box in a cool and dry place. Of course, I also have local backups.
Does anyone know if iCloud (photos) storage is AWS S3? Curious if using both iCloud & Amazon Photos is a decent redundant backup plan in addition to a local copy. Or if I have iCloud & Prime, why not? (I also use Backblaze.)
I think Apple uses GCP as well. A photo is probably replicated twice across AWS and GCP by Apple, and AWS and GCP will then replicate it at least 5 times on their own side onto different racks or even AZ.
What if a strong solar EMP wipe all storages on earth? you need redundancy on at least two other planets (I would suggest the moon but it's close to earth)
Anyone have a good iCloud backup option? I finally "gave in" and stopped manually pulling photos off my iOS devices and subscribed to iCloud. However I'm worried I'll get screwed or mistakenly delete something.
Install iCloud app locally and sync them. Then backup your local folder to another device or cloud. Backblaze B2 is a great cloud option for raw storage.
Does icloud drive contain the photos? My issue with icloud was that as far as I was aware, I could only sync photos via the Photos mac app, or web page downloading a zip archive a few gb at a time. Both options ended up failing for me unfortunately. The Photos app would quit instantly when I set it to sync the library because I guess it was too large perhaps. The web page downloads would time out a few hundred mb in. I tried using third party command line tools for icloud downloads, and they too would time out.
Exactly what I do. I use an old Mac Mini with an attached 5 TB USB drive. This provides a mirror of the iCloud data. For versioning, add a separate drive and use Time Machine. This provides versioning. For offsite backup, B2 or other backup provider.
Currently searching but don't get in this boat with me. I'm dealing with a library of 50k+ photos I can't get off icloud currently. All options on the table have failed after multiple attempts (Photos app sync, web page download, third party command line downloaders).
The conflation of sync and backup can actually be really dangerous. You don't want a primary backup to have the opportunity to delete files because they (apparently) have been deleted on the primary.
iCloud Photos will protect your photos in case if you lose your device or it fails or whatever. It won't protect your photos if you delete them (either accidentally or intentionally).
It is not a backup, and even "full" device backups WON'T include the photos, because of the (mostly correct) assumption that they are all in the iCloud Photo Stream. They will be backed up in "full" backups ONLY if you don't use iCloud Photos.
A 'better than nothing' but very easy thing I do is use two cloud providers for photo storage. Since my photos are all phone based, backing up to any cloud provider is pretty straightforward.
wat? SSDs aren't quite to 10TB yet, at least consumer ones. Do you mean multiple SSDs in an enclosure? Because I sure hope you didn't end up with a fake. Fakes have hacked firmware that makes it report the wrong capacity, and you only find out when you go to retrieve your data.
Additionally, I've heard SSDs aren't great long term backup mediums, compared to mechanical hard drives. I personally use both.
it's not common, but not very rare either, that we go to view an old set of pictures on Google Drive and find some of them are corrupt and cannot be displayed. I've never reported anything to Google, because I don't really have any evidence to provide. All I know is, we didn't upload all these 'corrupt' photos 5/10/15 years ago.
and I do not have backups of them all (which is on me)
1.5 Terabytes? (Assuming 6Mb/photo which is what I'm getting from an old nikon in raw format, dunno what everyone else is using - enlighten me please if necessary).
Less than USD$50 for a 2TB disk. So buy 3, post one to a family member when its full. Double the price it for SSD?
How much Amazon Photos storage do you get for $150? Ok for this guy, none, and I get connection refused from the link above so I got no details. But assuming it worked as advertised how much would you get?
Nikon RAWs are more like 50 Mb on their current cameras. Amazon offers unlimited photo storage with Prime.
I agree they should also have their own physical backup if the photos are important, even at ~10x the storage requirements you estimate, but I'd also say any complaints about Amazon losing their photos are valid.
Data point: I have 7T of "photo data". I think some proportion of that is video, but probably no more than 10%. I am quite old and it does include scans of all the film images I've taken since birth, and all of one side of my family's film media scanned, and backups of all the cell phone images from family members' devices. From memory however most of the data comes from digital-era DSLR images which can be 30-40M each.
None of this data is stored in a FAANG company's DC.
Having used Amazon photos for a long time, I have had a bunch of issues. It is certainly (in my mind) not a replacement for Google photos, but is a possible second repository / backup.
Advantages: unlimited full resolution backup
Disadvantages: Poor facial recognition Unable to manually add tags Poor folder /album management Android app has stopped uploading photos and support hasn'tbeen able to fix it Upload from PC only using their app (which isn't great) Upload from PC has "lost" the database and subsequently uploaded at least 2 versions of the just under 1TB / 50k images / videos from my family
The result is that actually finding or doing anything with the photos in the storage is not very easy, but at least everything is backed up there!
PiBox w/ Mirrored SSDs -> Server w/ RAIDZ2 (at another wing of the house) -> Windows NAS -> Backblaze Unlimited
Multiple SSDs and HDDs would have to go down at the same time, and there are hourly snapshots up to a month in case a roll back is needed.
If you're relying on a third party in any case except total catastrophe, you're just waiting for that moment to come! This is why Backblaze is at the end, I depend on them when I have failed to provide availability myself.
I don't know if it's due to one of those weird GDPR blocking rules (I'm in Europe) but amazonforum.com doesn't let me connect. It just drops my connection:
http https://www.amazonforum.com
http: LogLevel.ERROR: ConnectionError: HTTPSConnectionPool(host='www.amazonforum.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10b551e10>: Failed to establish a new connection: [Errno 61] Connection refused')) while doing a GET request to URL: https://www.amazonforum.com/
Those blocks are tiresome. Like the eu is going to go after the eg alabama times or other small sites. I just vpn to usa and send them an email. Google etc...easy meat for gdpr, but for small site - why are you tracking me anyway?
I just compiled a list of american news sites, run a container with mullvad vpn always connected + dante to route the sites to different mullvad socks5 proxies, do the rest as passthru to my normal connection (with the help of a browser extension)
A few years ago my (very non-technical) mother asked me to make and send her a photo album of recent pictures. My wife and I had so much fun doing it that we kept doing it. Choosing the best pictures, arranging them and making nice layouts and titles itself is a fun walk down memory lane.
Now we have a shelf with about a dozen volumes of memories and we found we are far more likely to look at these than we ever were with photos in cloud storage.
I like to think that if I lost all the digital copies, I would still be happy with what I have in these bound albums. I highly recommend this to everyone.