That's not why VS is still 32-bit. Apparently folks at MS dont think the extra address space isnt worth it and think it would adversely affect performance [1].
In my own experience in migrating servers from 32 to 64-bit on Linux 12 years ago, 64-bit was around 10-20% slower than the 32-bit build. Dont how itd fare today (no longer at that job). Even though performance was demonstrably slower, 64-bit it was due to edict from the CEO.
For what its worth, neither of the services I was responsible benefited from the extra address space. Mem usage for bother services peaked at around 750 MB.
Rico has a point to a degree and every VS version since then reduced memory footprint and improved performance (mostly due to not calculating stuff up-front which may never be needed and doing it on demand).
Microsoft has also pushed since VS 2008 (!) for plug-ins and extensions to use an out-of-process model. I think ReSharper got the message in summer 2019 that it might be a good idea – and incidentally, a lot of their performance comparison of Rider vs. VS comes down to "Rider uses ReSharper in another process, so the IDE doesn't slow down when ReSharper does things", while they themselves still pursued the slower model in VS. Heck, until a few releases ago ReSharper was still using the old synchronous COM APIs for querying the solution after loading, leading to severe pauses when opening a solution or re-loading it after switching source control states.
Vanilla VS is plenty fast and not that memory-hungry. In my experience it's extensions that try to sync with IDE state where a lot of the waiting comes in and in-process extensions where a lot of the memory usage comes from.