Begin blocks execute during compilation, immediately after they've been parsed. They're compiled themselves, and execute within the Perl interpreter before it finishes compiling the rest of the containing module. So they don't really execute prior to all compilation, but they execute prior to the compilation of anything sequentially after them in the source.
I don't consider this to be 'true' non-terminating compilation then. Interesting cases to me would be ones where the syntax itself throws the compiler into an infinite loop, not one where you explicitly tell the compiler to execute an infinite loop.
"the syntax" and "where you explicitly tell the compiler" are the exact same thing. Syntax is, fundamentally, instructions to the compiler.
Some languages don't contain instructions that can instruct the compiler to loop or recurse (C, assembly, brainfuck). Some languages contain a sub-language or meta-language which can instruct the compiler to recurse (java, C++). Some languages use their native syntax to instruct the compiler, making looping or recursion trivial (Perl, Lisp).
It's non-terminating execution during compilation, similar to the example elsewhere in the comments using nonterminating macro definitions in Common Lisp.
A compilation process that involves executing and waiting for the termination of a program that doesn't terminate can't itself terminate.
Perl is a dynamic language and can execute Perl during compilation. Some languages such as C++ allow metaprogramming via templates or other special compile-time meta-language constructs. Perl simply allows metaprogramming via Perl.
From the reception to my comment it seems HN readers aren't very familiar with dynamic languages. Perhaps I invited those subject to dunning-kruger by providing an overly simplified example.
perl -wle'BEGIN {1 while 1}'
Heh.