The N9 is easily worse than every other phone I have used and every phone anyone I know has used. I feel very silly for paying for it. Can you introduce me to someone who shares your opinion of it and does not have one yet?
From the home screen, hitting "Music" -> "Songs" involves about 20 seconds of waiting before you can pick "Shuffle." The ssh app is significantly worse than similar apps for other OSes because it is missing important keys. Putting music on the device took a couple hours because it would repeatedly spend a great deal of time indexing the music before telling me that I did not have any music. The UI is generally less responsive than is tolerable, even when doing extremely simple things like dialing a phone number. I do not enjoy going to the phone app, hitting a few numbers and waiting a few seconds before the phone makes any sound to assure me that it knows I am hitting numbers.
The swipe gesture is really cool until a bunch of apps also want to use swipe within the app. At that point, you get to play the fun game of trying to perform a swipe that is long enough to get the app to acknowledge it without starting at the edge of your screen, because that would cause the OS to switch out of the app.
There is no option to display 24-hour time and YMD other than by setting the locale. My phone now thinks that I am Japanese because I want it to display 24-hour time and YMD, but I do not know any Japanese, so every bit of software that localizes based on my locale is somewhat difficult to read.
It took a couple days to get IMAP IDLE to work, but even once it worked it would often repeatedly attempt to download a huge amount of email, fail midway through, and start again from the beginning rather than keeping whatever portion it had finished.
It is also missing a lot of software that two other OSes have, but I at least knew that would be true before buying it.
You'll be happy to know (or sad, depending on how you look at it) that your N9 is defective. It takes me 5 seconds at most from the home screen until I can hit shuffle, with all the loading, animations and whatnot, I never have the feeling that I am waiting. I also used the included USB cable to transfer my music to the phone, can't say I have experience with the ssh app.
The fact that you hit numbers in the dialing screen and have to wait a few seconds before the phone makes any sound is another strong indicator that your phone is broken and you need to have it replaced under warranty. It is not supposed to be behaving like that.
I have not seen any swipe conflict between the applications and the phone's UI, which is basically because for the built-in gestures you swipe from the edges, whereas for the apps you swipe from inside the window. I don't see how you can get a conflict out of that. Can you give me an example of app that does that? Seem like an app fault, not phone fault.