Things get a little better if you use the new Web Crypto APIs. They can act kind of like a virtual hardware security module from a web page's point of view: with the web crypto API, you can create and use an encryption key (symmetric or asymmetric) whose key material can never be exposed to javascript. The browser keeps the key material completely private from the web page, but lets the web page use the key for certain crypto operations.
This means that if a website uses this and generates a key through the Web Crypto API on the first access, the user only needs to trust the site on the first access (to serve javascript that actually uses the Web Crypto API) in order to trust that the key material stays safe. (However, if the website admin turns evil and wants one of the user's files to be decrypted, they could serve javascript to the user that silently makes them decrypt the file for the admin, so the problem isn't completely solved.)
This means that if a website uses this and generates a key through the Web Crypto API on the first access, the user only needs to trust the site on the first access (to serve javascript that actually uses the Web Crypto API) in order to trust that the key material stays safe. (However, if the website admin turns evil and wants one of the user's files to be decrypted, they could serve javascript to the user that silently makes them decrypt the file for the admin, so the problem isn't completely solved.)