Conversely, on public transit you can engage in other activities. When I take the train to work, it takes twice as long but I'm able to catch up on e-mails, take a nap, read a book, watch a show, etc. That's not possible while driving so I view it as a net time gain.
I think it comes down to preferences. I have never enjoyed reading, podcasts, or whatever else on cramped public transit, but I do enjoy the alone time of driving and I generally enjoy driving if it's not stop-and-go.
That said, the overall amount of time I spend is still my first priority. I don't want to spend any significant portion of my life going to and from work. I live in Chicago so public transit is 100% the way to go since it's faster and many times cheaper than using a car.