Chrome had a similar fuckup in April 2018. They changed the minimum granted quota for IndexedDb. Suddenly, all Android devices with low free storage space (< 1GB roughly) had a granted quota of 0. This of course led to all Android WebView apps relying on IndexedDb to have no access to existing stored items and no way to save new data.
It took them 6 weeks to push a fix. I used the opportunity to switch to Sqlite and can only recommend others the same.
This actually hit me really hard. I made a PWA I used to track certain stats but the data wasn't so important that it needed to be stored in a database.
Instead I had an import/export for manual backups and assumed that the data inside indexedDb wouldn't be dropped.
Yes, same for me. Also a bad assumption I made: People probably have enough storage space. The number of people running around with older devices that have 16 GB of internal storage is still like 5% or so. Add 32 GB devices, probably 10%. And most of them are constantly full.
Btw. the same is true of older iOS devices. They are constantly almost out of space and there is not enough free space to update to the most recent iOS version. It'd probably work with iTunes, but many people don't install iTunes. So they're stuck with iOS 10, although their device would still be capable of running iOS 13, simply because they don't have enough free space.
> So these browser storage mechanisms must be considered cache, not primary storage of important data.
Maybe I’m an exception, but I’ve always viewed these storage mechanisms as unreliable. I primarily use them for storing things like user filter settings on a list view. It never even occurred to me to use them for anything critical.
That seems like an uncharitbale reading. A manager let people know that they'll be able to tell if a new chrome release will affect them by testing with beta. The overwhelming sentiment was "we're sorry".
When Google introduced Chrome in 2008 (via comic!), they claimed that within 20-30 minutes of a beta build, it was automatically tested on tens of thousands of the most popular webpages. Is that no longer true? Does that not include any which use local storage?
AFAIK it is still true, but this bug is specifically with upgrading. Basically all the testing occurs with an empty profile; migrations are specifically undertested.
Also note that this bug only affected WebViews and not Chrome. Not sure how testing of migrations compares between the two.
I was bit by this in a cordova app on a past update. After that I backed up api keys, etc to native storage using a cordova plugin and added a wrapper that will fall back to it if localstorage lookups fail.
I would advise others to do the something similar since unless you are packaging the browser too (like crosswalk did), then that piece is really out of your control and you shouldn't expect it to be retained.
A similar issue can be seen on iOS under low storage conditions where the operating system will clear the browser cache to reclaim space.
Yes, that was clear. You say it uses localstorage, and only fallback to native when localstorage is not available. This is not the scenario of the bug. Here, localstorage is available, it just has been wiped out.
i still don't understand the core issue: why is anyone saving essential data to localstorage? isn't that supposed to be a state-related, somewhat temporary (due to its nature) storage in-between the UI and the API/database?
don't you lose all your data when you use a different pc or smartphone?
i see a lot of mentions of cordova apps. are people publishing apps without a backend (and save everything in localstorage)?
i've worked on pwa's in the past but server sync was always a priority in my algorithms.
does anyone have any realistic use cases where localstorage is so essential that clearing it would somehow lose tons of data?
Depends on the context. I've built Cordova/Phonegap apps in the past where you want things to work offline and sync once the user goes online. If the time from user going offline to online again is too long / device storage too little, that data that would eventually need to be synced, would be lost.
Always ways of working around it, like saving to files or whatever, but localStorage is a nice interface that works both for Cordova and client-side web apps.
agree. for the use case that you mentioned there would be a problem.
but as you said, it could be overcome by using native apis that store state. it's even imperative in case of secrets, like access tokens and the like.
for client side web apps there's a bigger problem with localstorage. from my experience end users switch browsers and platforms. and they expect consistency from their web sites/apps. this expectation is much more intense than for native apps. so, while practical for developers, i'm not sure it's that welcomed in web land except for always having a local backup or as a temporary storage to alleviate network issues.
but since every device is as persistent as its weakest point (e.g. physical damage, loss/theft, etc) shouldn't apps that need to save essential/important data basically give up the argument of persistence because of using localstorage in the first place? your data is one breakdown/loss away from permanent destruction when using localstorage exclusively.
that is why, from my experience, you always rely on the server and only use localstorage as a backup in case the server is down, there's no internet, etc.
claiming massive loss of data because localstorage got emptied is basically saying "our app wasn't designed correctly".
If my phone breaks, I lose my data, and that is my responsibility. You are saying that everything should be backed up on the cloud, I think that's crazy.
localstorage disappearing overnight IS a bug. If I put stuff in My Documents folder on Windows and Windows decides to delete it, that IS a bug too. You could argue that I should have backups, and you would be right, but that's no reason for Windows to delete my files, and that is no reason for me to never use my hard disk.
of course not everything should be backed up to the cloud.
but if it's important and only available on your phone, then yes please back it up, or better yet, have it up there in the cloud.
regarding your comparison between a windows folder and localstorage feels a bit exaggerated.
yes, the Chrome team really fumbled it.
and yes localstorage and important data should be mutually exclusive.
You're right that important data needs to be backed up, but I disagree that it is the app builder's responsibility to do that. It is the end-user's responsibility to think about backups, to choose where to backup and how often. From the app developer's perspective, it is commendable that, in this day and age, they allowed users to retain control of their own data, instead of automatically uploading the user's data to the developer's server.
Local storage and/or SQL DB were touted as features for exactly this use case. That Google did not consider their persistence as that important for the release is perhaps not surprising for a company used to storing all your data in their cloud (not claiming they did so knowingly, just that it's probably in their culture, just as in your comments above).
...localstorage and important data should be mutually exclusive.
It seems like we could say "don't trust this" about any aspect of any platform. Categorizing every application as "really important, don't update the UI until securely written to three AZs" and "totally ephemeral purge every minute" excludes a lot of stuff in the middle. Sure, nothing made by humans is perfect, but many things could be improved. One improvement to localstorage would be to simply not delete all the data.
Are you serious? Maybe you missed the entire privacy conversation around cloud storage in the last couple of years. Also, with your reasoning, your entire laptop should merely be a client for a remote storage.
Saving stuff locally is good for privacy, for performance, for offline-capabilities. You can back up your entire phone LOCALLY, to have a solution of your stated problems (theft, physical damage).
i am very serious. people lose their devices all the time.
never save important data on a single device. you simply delay the inevitable "i lost my data" scenario.
non-important data on the other hand should have no problems living on a single device. or even localstorage.
and as i said before, i don't think your average user has any ideas about backing up. that's why it almost always comes back to the developers to implement it.
I wrote a PWA which functioned as a calories counter and exercise tracker.
I stored the data on the client using indexedDB, offering an export/import option for manual backups.
I wasn't interested in the data and it's simpler to just store it on the client rather than setting up the infrastructure required to centrally persist this information.
It's really unfortunate that it's not a reliable storage medium.
Just how fast are Chrome rollouts on Android? Given how destructive this appears to have been, it seems amazing that they got to a 50% rollout before the first bug report.
>I heard from a company that uses localstorage for offline no-connection available that had local record of animals getting vaccionation. The update "erased" all the data. They don't know which animals got vaccine and can't repeat on all of them. Serious stuff.
Ideally, a backup solution on the OS would target the browser profile itself at regular intervals, and you could allow the user to export the data into a file that can be downloaded by the browser (or copy/pasted).
It should be obvious that any form of browser storage is unreliable, and MUST be treated as such, and it should be EXPECTED to be deleted at a moments notice - setting such expectations encourages good practice at backing up your data incase of a failure.
Don't rely on others to ensure business critical data is stored and accessible - that's partially on the users/application to offer what they can.
This has been happening in Chrome extensions for years now. I’ve tried reporting it so many times. In our case it happens randomly so it’s impossible to reproduce, which inevitably leads to ”issue closed - couldn’t reproduce”.
Even then, it can still be used for offline data... you collect/perform tasks without an internet connection and it syncs later. Chrome updates before data sync, boom, all your data is gone.
Now, as for years of data, I mostly agree, this should absolutely be backed up and/or synced more regularly.
That still seems like a poor design. This would also happen if you cleared localStorage before the data is synced. If, for example, when you came back online and were trying to do something on an unrelated site, hit a bug and tech support asked you to clear your browser.
For something as critical as financial records or vaccine records, the sensible thing might be to not work in offline mode, or to persist to native storage.
I think it's fine as a cache for offline data for a lot of use cases (i.e. task trackers, note apps, shopping lists, etc). But it creates an unacceptable risk for business critical data, imo.
Don't you have the same problems with native? It's like like no OS update ever broke apps, or that Apple doesn't capriciously remove APIs, or reject you from the app store sometimes for unexplained reasons.
Hell, people get broken by OSS ecosystem libraries a lot.
True to some extent, though I don't think there have been many OS updates which deleted all storage for all apps (well, moved to an inaccessible place)... This really is a massive failure of QA and dev.
Wish Apple and Google would put more browser collaboration time in, for the benefit of all..constant one upmanship hurts both and millions of man hours in bugs, pollyfills, API repairs, and on and on. Webview, WKWebview teams need more rigorous standards and safety checks clearly for consistency and reliability. Progressive development benefits both. Web-apps are a critical, large and important part of the app developer ecosystem.
Indeed, I don't mean to downplay the frustration of the developers and their customers. However, valuable data should really not be kept on a single mobile device, which is prone to damage, loss or stealing.
Wrong. Relying on being allowed on some appstore is much more like being "at the hands of very fickle gods". Pray that apple/google don't suddenly change their mind and lock you out, because it competes or goes against whatever new trend they adopt this year.
I'm linking Chrome issues here because Chrome is the most visible of all browsers with the largest marketshare, but other browsers have their issues, too.
Just to set the record straight on the part I'm familiar with, "Custom Elements V0" was never a web standard. It was a proposal, which Chrome implemented and shipped without getting everyone else on board, and which turned out to have significant enough problems that a somewhat different thing got standardized and shipped in all browsers ("Custom Elements V1").
Once something is shipped, people will rely on it. For example, quite famously, Youtube did for a very long time (until this year IIRC). AMP used to use it.
There are several polyfills available. Several Polymer versions targeted v0. Etc. etc.
It's not specifically Google's fault that they are removing support for these, but, once again, that's the state of the world: browsers are lrage complex platforms that may and will break your app.
Sure. Unshipping things is always hard, and likely to break people. But there's still a difference between unshipping support for a standard and unshipping support for a non-standard technology that only you ever implemented. In the latter case, people should be a lot more wary of depending on the technology in the first place.
While it's true that browser updates may break your app, it's also true that there are ways to write the app so as to minimize the risk of such breakage. Using Custom Elements V0 was _not_ a good way to avoid potential breakage...
It took them 6 weeks to push a fix. I used the opportunity to switch to Sqlite and can only recommend others the same.