I had to think to recall the reason. I had a support ticket at Fastmail about this in 2024 - I wanted to host my own CardDAV server and have Fastmail use and sync to this. This was not possible - I was told that two-way sync with other services was a "calendar only feature". I might re-visit.