An unconference (like us) can get away with that. A "real" conference often cannot. While certainly not every speaker should be paid, having known names for keynotes is important. Even just comping travel gets very pricey for small conferences.

Also remember in the scifi, anime, and comic con worlds, many of those people who offer to speak for free are doing it as a marketing expense. Nothing wrong with that, but it should be recognized because, depending on your conference, it can alter the flavor. We've been very explicit: no direct marketing.