But additionally, I like how it's not simply "pushing to the top", it does shows a previously selected language on the top, but it still keeps it duplicate in the list below, in case the user is going by muscle memory.

To me this is the best way.

Either make it VERY OBVIOUS that you're removing the item from the bottom of the list (which wouldn't be possible here), or don't remove it at all.

If I had a cent for each time a SaaS made my life harder by trying to "help me" I would be CEO of every SaaS I use.