KVM VPSs are generally significantly more expensive than OpenVZ, so for side projects where I want a cheap (<$5/mn) plan I've always used the OpenVZ VPSs.
You can use the go implementation of wireguard (wireguard-go). This way you don't need a kernel module. Works for me on a really cheap OpenVZ VPS since months.
