My mum had a mechanical typewriter and yes it would do that. However English has no formal diacritics. We do have shit loads of borrow words from French and others that do have them.
If the back tick on the en_GB keyboard is a sop for the grave accent, then why do we not have an acute accent available as well? or a cedilla for that matter?
So if you pressed backspace followed by "`", you'd give the last character a grave accent.
That's my theory for it being called backtick anyways.
Nowadays you could configure your input method to make that key give the next character a grave accent