Was it a 1U server? 2U and 4U servers can be typically be configured or modified to run at a noise level comparable to a gaming desktop. Performance 1U servers you'll hear through the floor though for sure.
I can tell you my 1U R620 servers lasted less than 6 months before I sold them for 2U R720s. I still wouldn't want to sit in a room with the R720s but you couldn't hear them through a door. Sold those when I moved and downsized to a tower server R430, equivalent to a 4U. It's in my office now and no louder than my desktop. I'll get rid of the raid array and downsize to a single hdd at some point to silence the heads clicking though.
I had a single SuperMicro in the basement one summer and it had to go after a month or so. You could hear it everywhere.