> Here's a Python program that contains no statements: 42.
It is true that Python's grammar features a statement/expression dichotomy, unlike many other tools. If we want to speak to Python's grammar, we should make sure to consider two general eras—before and after the introduction of the PEG parser. The PEG parser was introduced to Python with PEP-617[3].
Let's consider first the grammar used in Python 3.8, prior to the PEG parser. You can find this in Grammar/Grammar[4]. As we can clearly see, there are a number of “entry points” for a well-formed Python programme[5]:
We can see from the above that, with the exception of `eval_input`, we consider a well-formed Python snippet to be a sequence of statements. The programme snippet `42` would be parsed as an `atom` which forms an `atom_expr` which is part of a rightward-chain that begins with `test` which eventually rolls up to `expr_stmt` where a `testlist` is considered to be an a . This elides a number of details (because, for most users, even this simplification is exhausting and useless) and may itself be slightly incorrect, but it illustrates that, as another poster asserts, the CPython reference implementation grammar prior to the PEG parser considers single expressions in the context of a `file_input` to be `expr_stmt`—expression statements.
The Python parser considered a file input to be a “sequence of statements.”
But, of course, who cares?
Remember that the goal of an instructor is to present just the right level of detail that an attendee can do something useful. It is the case that the expression/statement dichotomy is useful, especially when considering the common expression⇋statement dualities we see in the grammar, but this is not a topic for day one, slide one of an intermediate/advanced course.
By the way, if we look at the PEG grammar, we see similar[6]. A `file[mod_ty]` input is comprised of `statements` (and an `eval[mod_ty]` input is comprised of `expressions`.)
Therefore, it is incorrect to say that “a Python program may contain statements or expressions.” Instead, we should say that the Python interpreter parses a simple, single-file Python program as a sequence of parser-level `statements` which my themselves be value-producing entities (which we might refer to as “expression”) or non-value-producing entities (which we might refer to as “statement.”) We might note that there are places where “statement” grammatical entities are invalid to use and places where “expression” grammatical entities are invalid to use, and that this “statement”/“expression” dichotomy is one that is found in other programming languages (but that there are other programming languages which are avoid this distinction.) We might further note that this dichotomy has affected evolution of Python be introducing dualities—for all but a few “statement” forms, there is an equivalent “expression” form. There may be contortions required to exactly match one to the other. (e.g., `while`) There are cases where, absent significant contortions, there is no dual (e.g., `try/except/finally/else` or `match/case`.) As a consequence, there is not considered to be a simple way to transform any multi-line Python programme into a single-line, single-expression equivalent. But, of course, probably nobody really cares. You've lost the class on slide one.
Next, it is true that semicolons can separate Python statements in some cases. It is important to note that, with the exception of silencing last-expression output in a Jupyter notebook, it is possible to never encounter the use of a semicolon in real Python code.
It is unfair to assume the author intends to convey that the execution of a Python programme is strictly in the order of the appearance of the lines of code in a file, without considering that function calls contain a body of statements which are executed only on function evaluation. Instead, we should interpret this to bullet point to mean that Python programme executed top→down with statements executed at runtime in a manner dissimilar to how C++ works. For example, `def f(): pass` is executable code in Python, and this statement is, in fact, executed. This is why we might argue that the “mutable default argument” problem is largely a matter of misunderstanding Python's execution model. We could consider that the “execution” of the `def` starting statement means the parsing and compilation of its contents into bytecode, rather than the execution of the contents directly. After all, even `f()` on `def f(): …` is not guaranteed to actually evaluate the body of `f` in all circumstances.
It is incorrect to suggest that the presence of features like `sys.meta_path` or `sys.path_hooks` invalidates the top→down nature of Python's execution model. Sure, you can implement an import hook that `exec(''.join(open(…).readlines()[::-1]), …)` but this could easily be considered a modification of the input rather than a modification of the `exec` mechanism. In fact, there are many ways to generate bytecode without passing through the `exec` machinery, but these are generally considered esoteric. If we consider only the `exec` machinery, then we will see that it will invoke the standard tokenisation and parsing process which will execute its payload line-for-line from beginning-to-end. Sure, we can say that the presence of import hooks mean that a programmer could subject a file input to preprocessing prior to passing it to the `PyRun_`/`PyEval_` mechanisms or that a programmer could generate executable bytecode in any arbitrary fashion bypassing these mechanisms… but it's going to be rare to encounter these… on the first slide… on the first day… of an intermediate/advanced Python course. So who cares?
This has already been pretty exhausting, but I think I have adequately demonstrated that it is not the case that “every statement [here] is a lie” and that we can casually dismiss the work of David Beazley as that of a “witch doctor” or “snake oil peddler.”
In fact, I would suggest that I have put forth some evidence that the original criticism belies a weaker and less thorough understanding of not only the Python interpreter but of the instructional process than its confident tone might suggest. I hesitate to suggest further.
It's definitely unfair to cast aspersions on the CPython core development team. Python is an established, mature language with a growing, increasingly diverse development team. It is bound to be the case that there will be new contributors who do not have as strong an understanding of the entire language and the entire interpreter. It is not the case that the CPython core development team is as untalented as you suggest. In fact, I would assert that there are programmers among the CPython core team who are some of the most talented people I have ever met. Many are experts not only in Python, but in C, in C++, and across all languages that they use in their work.
> The Python parser considered a file input to be a “sequence of statements.”
You missed the whole point and decided to answer a completely different question. Intentionally. Because you don't like the inconvenient fact that OP wrote something idiotic. OP wrote in their course that Python runs a program that is a sequence of statements. Which is nonsense. Python doesn't run or execute statements at all. This is not how it is implemented, nor was ever intended to be. Statements and expressions are read not run.
Now, for some reason you decided to ignore the part where Python is executed interactively, and conveniently found that in some other context 42 is a statement? -- well, so what? Python the implementation, the documentation, the infrastructure -- all of this is made by incompetent programmers. It's all bad. They ended up putting what they claim to be an expression into a statement in the parser because that's how it was convenient to write the parser? -- well, who cares? This shouldn't be the test for when something is an expression or a statement. Expressions evaluate, i.e. have a defined evaluation result, statements don't. That's all you need to know when describing the difference. Your archeological digging into a pile of dung that is Python implementation are misplaced and only misguide those who'd like to know anything about this language.
> to mean that Python programme executed top→down with statements executed at runtime
This is a pile of absolute horseshit. No, Python doesn't execute statements. Stop writing nonsense. Statements don't exist at runtime. What the hell is even this "top->down" you are talking about. Do files have top and bottom in your world? What is the top of the file?
Very verbose and obfuscating reply to a comment that has been censored and that we cannot see.
It is symptomatic for the Python space that criticism is drowned out by consultants, who write loads of perfect English and focus on the irrelevant.
The only part I'd agree with is that Beazley is competent. Has he written and maintained production ready code though? I wish that people who actually do something would also receive such defenses.
But no, in the Python space the trainers and consultants are superior to the people doing the work. and they rake in the cash.
> Here's a Python program that contains no statements: 42.
It is true that Python's grammar features a statement/expression dichotomy, unlike many other tools. If we want to speak to Python's grammar, we should make sure to consider two general eras—before and after the introduction of the PEG parser. The PEG parser was introduced to Python with PEP-617[3].
Let's consider first the grammar used in Python 3.8, prior to the PEG parser. You can find this in Grammar/Grammar[4]. As we can clearly see, there are a number of “entry points” for a well-formed Python programme[5]:
We can see from the above that, with the exception of `eval_input`, we consider a well-formed Python snippet to be a sequence of statements. The programme snippet `42` would be parsed as an `atom` which forms an `atom_expr` which is part of a rightward-chain that begins with `test` which eventually rolls up to `expr_stmt` where a `testlist` is considered to be an a . This elides a number of details (because, for most users, even this simplification is exhausting and useless) and may itself be slightly incorrect, but it illustrates that, as another poster asserts, the CPython reference implementation grammar prior to the PEG parser considers single expressions in the context of a `file_input` to be `expr_stmt`—expression statements.The Python parser considered a file input to be a “sequence of statements.”
But, of course, who cares?
Remember that the goal of an instructor is to present just the right level of detail that an attendee can do something useful. It is the case that the expression/statement dichotomy is useful, especially when considering the common expression⇋statement dualities we see in the grammar, but this is not a topic for day one, slide one of an intermediate/advanced course.
By the way, if we look at the PEG grammar, we see similar[6]. A `file[mod_ty]` input is comprised of `statements` (and an `eval[mod_ty]` input is comprised of `expressions`.)
Therefore, it is incorrect to say that “a Python program may contain statements or expressions.” Instead, we should say that the Python interpreter parses a simple, single-file Python program as a sequence of parser-level `statements` which my themselves be value-producing entities (which we might refer to as “expression”) or non-value-producing entities (which we might refer to as “statement.”) We might note that there are places where “statement” grammatical entities are invalid to use and places where “expression” grammatical entities are invalid to use, and that this “statement”/“expression” dichotomy is one that is found in other programming languages (but that there are other programming languages which are avoid this distinction.) We might further note that this dichotomy has affected evolution of Python be introducing dualities—for all but a few “statement” forms, there is an equivalent “expression” form. There may be contortions required to exactly match one to the other. (e.g., `while`) There are cases where, absent significant contortions, there is no dual (e.g., `try/except/finally/else` or `match/case`.) As a consequence, there is not considered to be a simple way to transform any multi-line Python programme into a single-line, single-expression equivalent. But, of course, probably nobody really cares. You've lost the class on slide one.
Next, it is true that semicolons can separate Python statements in some cases. It is important to note that, with the exception of silencing last-expression output in a Jupyter notebook, it is possible to never encounter the use of a semicolon in real Python code.
It is unfair to assume the author intends to convey that the execution of a Python programme is strictly in the order of the appearance of the lines of code in a file, without considering that function calls contain a body of statements which are executed only on function evaluation. Instead, we should interpret this to bullet point to mean that Python programme executed top→down with statements executed at runtime in a manner dissimilar to how C++ works. For example, `def f(): pass` is executable code in Python, and this statement is, in fact, executed. This is why we might argue that the “mutable default argument” problem is largely a matter of misunderstanding Python's execution model. We could consider that the “execution” of the `def` starting statement means the parsing and compilation of its contents into bytecode, rather than the execution of the contents directly. After all, even `f()` on `def f(): …` is not guaranteed to actually evaluate the body of `f` in all circumstances.
It is incorrect to suggest that the presence of features like `sys.meta_path` or `sys.path_hooks` invalidates the top→down nature of Python's execution model. Sure, you can implement an import hook that `exec(''.join(open(…).readlines()[::-1]), …)` but this could easily be considered a modification of the input rather than a modification of the `exec` mechanism. In fact, there are many ways to generate bytecode without passing through the `exec` machinery, but these are generally considered esoteric. If we consider only the `exec` machinery, then we will see that it will invoke the standard tokenisation and parsing process which will execute its payload line-for-line from beginning-to-end. Sure, we can say that the presence of import hooks mean that a programmer could subject a file input to preprocessing prior to passing it to the `PyRun_`/`PyEval_` mechanisms or that a programmer could generate executable bytecode in any arbitrary fashion bypassing these mechanisms… but it's going to be rare to encounter these… on the first slide… on the first day… of an intermediate/advanced Python course. So who cares?
This has already been pretty exhausting, but I think I have adequately demonstrated that it is not the case that “every statement [here] is a lie” and that we can casually dismiss the work of David Beazley as that of a “witch doctor” or “snake oil peddler.”
In fact, I would suggest that I have put forth some evidence that the original criticism belies a weaker and less thorough understanding of not only the Python interpreter but of the instructional process than its confident tone might suggest. I hesitate to suggest further.
It's definitely unfair to cast aspersions on the CPython core development team. Python is an established, mature language with a growing, increasingly diverse development team. It is bound to be the case that there will be new contributors who do not have as strong an understanding of the entire language and the entire interpreter. It is not the case that the CPython core development team is as untalented as you suggest. In fact, I would assert that there are programmers among the CPython core team who are some of the most talented people I have ever met. Many are experts not only in Python, but in C, in C++, and across all languages that they use in their work.
[1] https://github.com/dabeaz-course/python-mastery/blob/main/Py...
[2] https://github.com/dabeaz-course/practical-python/blob/maste...
[3] https://peps.python.org/pep-0617/
[4] https://github.com/python/cpython/blob/3.8/Grammar/Grammar
[5] https://github.com/python/cpython/blob/1663f8ba8405fefc82f74...
[6] https://github.com/python/cpython/blob/main/Grammar/python.g...