Of course having source is more convenient, but not as much as it may seem. The attacker is looking for something the source did not intend to do, so they're already "reading between the lines".
Many memory safety bugs can be found by fuzzing code as a black box. Fuzzing is used by code authors too, because even people who wrote the source code don't fully understand what edge cases exist based on the source code! If the code has Undefined Behavior bugs, then the source code may not even match what the actual program does.
There are good decompilers, and as I've mentioned, writing an exploit will necessary depend on working with compiled code — you must know what's in memory, on the stack, and where "gadgets" are the exploit can jump to. This information is not present in the source code. Deep understanding of compiled code is a prerequisite for writing an exploit.
Bugs have been found in closed-source Windows for as long as it existed. Even the recent attack on Apple Messages combined this bug with a bug in Apple's closed-source sandbox. Security by obscurity has always been tempting, and never worked as well as hoped.
