Hah I just realized I could check the kernel config in a vm. Earlier I looked for a kernel package to install in a container to see what the config was. Maybe I've been drinking too much. Anyway, in one of my Alpine virtual machines, /boot/config-virt has CONFIG_TLS unset.
The article actually mentions Alpine as one of the dist. where it's not supported by default:
>The following OSs do not support kTLS, for the indicated reason:
>Alpine Linux 3.11–3.14 – Kernel is built with the CONFIG_TLS=n option, which disables building kTLS as a module or as part of the kernel.
and even recommends building OpenSSL and Nginx 3.0 yourself anyway, so looks like it will be a while before this might be available out-of-the-box for most major dists. But of course everything is OSS so you can DIY if you don't mind getting some ./configure under your fingernails :)