I get that, I just never thought it was very funny. Start might not have been the best name for that button, but the idea of having a single a menu to access all OS functionality and applications was good. Still is good, in fact, which why so many people got upset when they removed it in Windows 8.
If you think of shutdown as a "stop" command to the computer, you see the contradiction: to stop your computer, press start.