The folks at OpenBSD Misc do not recommend port knocking. IIRC one of the reasons stated was a possibility of getting locked out of your own server. I tried looking up the relevant thread, but am on the phone.
Thank you. One learns something new everyday. And I feel humbled by most comments I read here.
I did look up OpenBSD pf again, and it too does port knocking in the kernel. My information was dated (left the misc mailing list at least 5 years ago).
You can set an alert for every failed SSH connection because if someone is able to get through that, it's alarming.
This setup has the side effect of reducing your log noise to zero. That SNR is super important for intrusion detection.