They name their architectures after scientists (Maxwell, Pascal, Turing, Volta, Ampere, Lovelace, Hopper). Thats what the GPU initial stands for.
As for the number, the die name counts down to 100 (with GA107, for instance, being a small GPU die and GA100 being the big one), and the big datacenter GPU as a product inherits the 100.
Also, it occurred to me that Nvidia does sometimes increment the die to 200 (EG GM200, as the Maxwell 100 series was a single small oddball die). Its possible that they "refreshed" the GH100 die and are codenaming it GH200.
At first they used "measures of hotness", but ran out a few generations in. Coincidentally, degrees of warmth are all named after scientists. So they continued with scientists.
As for the number, the die name counts down to 100 (with GA107, for instance, being a small GPU die and GA100 being the big one), and the big datacenter GPU as a product inherits the 100.