No more annoying than Bluetooth pairing PINs or iOS's passcode-to-use-USB prompts. If the keyboard has secure stateful memory (e.g. for a client-certificate or client-secret) then the user would only have to enter it once.