Crystal suffers from other problems. They diverged too much from type annotations and use macros / metaprogramming too much which immensely increases the compile times.
I don't know about Crystal's preformance problems specifically, but I would be surprised if macros would take up much compile time. I would have guessed it was more to do with the fiendishly tricky type inference problem that they set for themselves (ie. global inference + subtyping + union types + inheritance + overloaded methods).