The only appealing thing to me would be that keyless entry/start usually comes with remote start, which in the northern climates would be nice.

But I'm with you as far as having a physical key goes. Never have I thought, "man, this key stuff is just so tough!" especially since every other door in my life still uses a mechanical key which would likely be attached to the remote key fob. When I get home and turn off the car I'm going to need those keys in my hand anyway to get in my house.