I'd say Japanese is even more difficult than Chinese - at least with Chinese the majority of characters are predictably one syllable with the same pronunciation regardless of context; in Japanese you get entirely different pronunciations depending on the context, with a ton of exceptions.

That said the actual sounds are few and really easy to vocalize (if you ignore pitch accent). Adults do use kana for certain words and in between words, but it's true that you do need to know a lot of kanji to read things fluently.