You're probably in much better shape than I am. For you, a 5000 foot day is probably fairly routine. For me, it still takes some willpower. I'm sure I will adapt over time.
But yes, to answer your question, I train at sea level and climb between to 6000-10000 feet on weekends, and sometimes high camp.
If your RHR is elevated at high camp that's to be expected as that's part of the physiological response to reduced partial pressure of oxygen.
If your RHR is still elevated after your return to sea level...huh.
I live at 6800 feet now but like you used to live at low elevation. I'd come out, hike up Pikes Peak (6700 feet start, 14100 foot finish) and then come back down to sleep at 6000ish feet. RHR would be up maybe 10-15BPM until I went home to sea level, then it went back down to normal literally on the drive home.
So the response you're seeing, I mean what do I know I'm not a researcher, doctor or expert but I will say that seems odd.
Check out this 40 minute podcast on altitude physiology. Super dense with really actionable information on the body's responses:
https://www.scienceofultra.com/podcasts/29
But yes, to answer your question, I train at sea level and climb between to 6000-10000 feet on weekends, and sometimes high camp.