Have you also set up reverse DNS and done it all for IPv6 too? The second part was what I was missing a while back. I do agree that Google and Microsoft are extremely strict with what they accept and it's not always easy to tell why you get thrown to the spam pile.
Yep, I think there was a post about setting up IPv6 correctly on HN a while back. Alternatively, you can make your host not respond to IPv6 at all.
If I remember correctly, the problem isn't that IPv6 needs to be set up, but that if your host listens on IPv6, Google will default to that and then your SPF/DMARC/DKIM setup needs to work with IPv6. If you don't have an external IPv6 address, then you don't need to configure SPF etc... for IPv6 and you should be fine.