I am not going to walk around all day in a headset. Everything you describe is AR, not VR. A HUD in my glasses would be fantastic but a completely different product than an immersive VR headset.
That's why I mentioned "much more refined devices".
To me it seems obvious that Apple made a headset only because that's the minimum amount of hardware needed, right now, for the functionality they want. What they really want is "Apple Glasses", but they're willing to set money on fire for however long it takes to get there, building out the full-featured software ecosystem along the way so that by the time the mass market likes the idea they've already completely conquered the niche.
