I have a system that I use to rate movies: "Must see", "Worthwhile", "Wait for the video" and "Don't waste your time." After watching Mulholland Drive I had to invent a fifth rating: "I want those two hours of my life back."
Lynch fills his works with characters and scenes that are—at best—plot-adjacent, whimsical/incongruous/uncanny, and full of often-unexplained and apparently-metaphysical mystery or magic. He's as likely to let mood or theme drive the content of and editorial-decision-to-include a scene or a shot or an entire character(!), as plot. It makes sense you'd dislike his stuff if you don't like the Bombadil section of LOTR.
(this is not intended as some kind of judgement, to be clear, I was just curious if works with similar qualities had the same effect for you, or it was something specific to that kind of element being in LOTR in particular, maybe because of the context or something)
It's both. I like a story to go somewhere, to have a point. I want it to at least be possible to figure out what the fuck is going on, even if it requires some effort and isn't obvious on a first reading (or viewing in the case of movies). In fact, the best stories are the ones where you have to do some work (The Godfather is my poster child). But if I put in the work, I want it to at least be possible that I could figure it out.
LOTR is mostly like that. Most of the events make sense in the context of the overall arc of the narrative. If you ask, "Why is this scene here?" there is usually an answer.
Except Tom. He appears, puts on the ring, fails to disappear -- which is quite extraordinary! --- and that's it. The end. Toodle-oo. Vanishes entirely from the story without so much as a by-your-leave. Why was he introduced? To this day I have no idea. And that is why Tom bothers me particularly. With Lynch you know what you're getting. Tom seems like a bit of a bait-and-switch.
Does that answer your question?