can't zstandard use standardized dictionaries as well?

Yes, sort of: "Zstd offers a training mode, which can be used to tune the algorithm for a selected type of data. Training Zstandard is achieved by providing it with a few samples (one file per sample). The result of this training is stored in a file called "dictionary", which must be loaded before compression and decompression. Using this dictionary, the compression ratio achievable on small data improves dramatically."

https://github.com/facebook/zstd#the-case-for-small-data-com...

According to the Bugzilla thread, Facebook won't bother going through the process of publishing standardized dictionaries.

But surely anyone could publish dictionaries? Like let's say Mozilla could publish a dictionary trained on HTTP headers, HTML and Javascript, and then accept traffic compressed with it under a "Accept-Encoding: zstd/mozilla" header

I wouldn't go to the trouble of determining the privacy implications of standartized dictionaries built from user data.

Brotli is super slow and thus suitable only for static content.

zStd also supports pre-trained dictionaries, it is particularly good for specific small JSON, for example. It won't be that helpful for relatively large HTML files.

The problem is brotli compression is pretty expensive/slow to encrypt. Which isn't really a problem for static content that can be compressed ahead of time. But doesn't work very well for just in time compression.

This isn't true. Brotli compression speed is similar to Zstd at comparable compression ratios.

https://quixdb.github.io/squash-benchmark/unstable/?dataset=...

Perhaps, this common misunderstanding is due to the fact the slowest level (-11) is the default one in the brotli command line utility?

It's also faster than gzip, and gzip was used for "just in time" compression on the web for ages, so I don't see how brotli can be a problem for this use case. (I assume by "encrypt" you mean "compress".)

On a project I worked on, we tried switching from gzip to brotli for on the fly compression, and it significantly increased CPU usage. That was years ago though so it's possible things are better now, or maybe the library we used just didn't support a compression ratio as low as what we were using with gzip.

Fair enough, I think there were some issues with early releases, but they should be fixed now.

I wish Caddy would support brotil, is either gzip or zstd currently.

Just did some quick research and it looks like you can serve pre-compressed static resources in Brotil. See the precompressed option of the file_server directive[1].

The thinking seems to be due to how CPU intensive it is, Brotil is not favoured for on-the-fly compression[2].

If you're really desperate for it though, you could try [this extension][3].

[1] https://caddyserver.com/docs/caddyfile/directives/file_serve...

[2] https://caddy.community/t/caddy-v2-brotli/8805

[3] https://github.com/ueffel/caddy-brotli

The extension is not performance focused, so its slow.

Other webservers support it, mainly Nginx. But since Caddy is written in go, they need a native go port which doesn't exist yet.

> The extension is not performance focused, so its slow.

Yeah, hence my "really desperate" qualifier. I probably should have been more explicit.

Caddy 2 supports brotli static compression out of the box, I ran a test on it just last week with this exact configuration. Dynamic, I'm not sure. That use case doesn't interest me as much.