I actually have the opposite experience, at least with the books I read. Nonfiction is typically highly repetitive and redundant so I find it suits audiobooks far better. Meanwhile fiction has a lot of subtle nuance and rewards carefully reading each sentence.
Of course this depends on what books you read in each category, if it’s a physics textbook vs Colleen Hoover then it’s the opposite.
For me this difference manifests in the speed I can listen to. I typically listen to podcasts at 1.7× speed whereas I listen to (fiction) audiobooks at 1× speed. Partly this is because it lets me appreciate a good reader's work better with intonation, different voices, etc., but it's probably also what you note that they're at different information densities.
Even so, I tend to rewind when I notice I missed something, e.g. due to my attention being needed elsewhere for a bit. I mostly listen while cycling and as long as I'm outside the city there's few distractions, but when sharing the streets with others there's often distractions that are more important than listening.
I've found this to be particularly true with William Gibson, tried to listen to some but a split second lapses in attention forcing me to go back often and I gave up. And just pick up a text.
Of course this depends on what books you read in each category, if it’s a physics textbook vs Colleen Hoover then it’s the opposite.