> Mypy will get rid of the Global Interpreter Lock (GIL)
>Details of the concurrency model are still undecided, though
Wait, what? There have been numerous attempts to get rid of the GIL, most of which failed miserably. A promise to get rid of the GIL without even deciding on a concurrency model seems a little premature.
This is not true. There have been plenty of successful attempts to remove the GIL, including from CPython. It's not impossibly hard to do so by a long shot.
The GIL is an optimization though, and so the performance hit from removing it on single threaded code is prohibitive. You can watch any of David Beazley's talks on the GIL for more info.
I stand corrected. Still, saying they are going to remove the GIL without addressing the challenges faced by other people who did the same - most notably the performance hit - makes me skeptical as to their ability to fulfill their promises.
Prohibitive is too strong I think. The patch to 1.4 discussed in a blog post by Beazley came with a 2x slow down, but that was an initial patch without much work on optimization. Besides that, I think 2x slowdown isn't that bad considering the language is not used for its raw efficiency anyway.
No the reason to remove it is to get parallelism from threads. Once you have parallelism, you can overcome a 2x slow down for a single thread by using more cores.
But 2x was just with the initial, non-optimized patch. A language like Java seems to perform quite well even with fine-grained locking, so it's possible.
And I don't see why performance is suddenly used as a deal breaker for a language primarily used for scripting and other non-CPU-bound purposes.
I speculate that the performance issue wasn't really the most important reason for rejecting the patch, but more so not wanting to deal with the complexity of maintaining the patch. It's a shame because the future is definitely going to have lots of cores, and not all problems are well-suited to multiprocessing.
The difference is that Java's memory model was designed from the ground up to work with multiple threads. Python's semantics are that only 1 thread is running at any point in time. Keeping that illusion while running multiple threads in parallel is what causes the huge overhead. The alternative is to do away with CPython semantics, but that would break a ton of existing code.
>And I don't see why performance is suddenly used as a deal breaker for a language primarily used for scripting and other non-CPU-bound purposes.
People want to use Python for lots of CPU bound processes. A big Python niche is numeric and scientific computing.
You say that it is "primarily used for scripting and other non-CPU-bound purposes", but this is also a consequence of it being slow. If it was faster it would open MORE uses. LUA and C# got the games action. Go gets attention on the networking apps end, etc...
We don't believe that it is practical to adapt the CPython VM incrementally for running parallel workloads efficiently. Instead, we are going to get rid of the GIL by having a new VM for running mypy code. The VM will be based on the Alore VM, but it will use compiled native code from the start (the Alore VM uses a bytecode interpeter). It will use a garbage collector instead of reference counting (CPython primarily uses reference counting), and it will also support a modified C Python extension API that will support proper multithreading without a GIL.
The CPython VM will still be used to run Python code and modules, and it will still be limited to running a single thread at a time (most of the time). Objects passed between the VMs will have to be copied or accessed using proxy objects, so there will be some performance overhead compared to native mypy modules. We plan to port commonly used, performance-sensitive Python modules to mypy and the modified C API to minimize performance bottlenecks.
