PHP is designed to be embedded in HTML. That is why you must use the enclosing <?php ?> tags in all PHP scripts. Even when you run standalone php scripts from the the command line, you're literally embedding PHP in STDOUT.
This is very different from other languages such as ruby where templating uses the ERB module, which is a subset of the ruby language.
That wasn't how the language was originally intended to be written, and that is not how hordes of people use it. But it can be done. Really.