Yes, yes, yes! This is what I've been waiting for so long.
Python has emphasized readability for the reference implementation vs. the practical benefits of better performance for everyone, and the community is really hurting for it.
Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better.
Python can be ahead in the performance race. We just need to get real.
> Python can be ahead in the performance race. We just need to get real.
I don't know if this is the case. Python's entire value proposition is that it is "executable pseudocode" with an extremely low barrier to entry - so somebody like a scientist or business analyst can solve problems without a deep understanding of computer science. Those goals are always going to run counter to performance, and python would have to re-invent itself at a foundational level to actually compete on performance with languages which were optimized for it from the ground up. Python's already winning in a lot of super relevant ways from playing to its strengths, and I don't think it makes sense to compromise those strengths to reach middle-of-the-pack performance.
If you ask me, if the python community really wanted to advance their interests, the thing to focus on would be dependency management and project encapsulation. If I could have a ux like npm or cargo for python, I would surely be tempted to use it more outside of jupyter. But it would not be because of performance ;)
You and the parent are mistaken about the cause of Python’s performance problems. Python isn’t slow because it’s readable—it could be quite a lot faster without compromising readability. Python is slow because it exposes the entire CPython interpreter as the C-extension API, which means they can’t change much about the interpreter without breaking compatibility with some extensions (and they are unwilling to do so). Since performance is so bad, the community leans hard on C-extensions, which worsens the problem.
I suspect one of the reasons packaging is so bad is because for each node on the dependency tree, you need to download and execute setup.py in order to discover the node’s direct dependencies, and to get the transitive dependencies you must download and execute the direct dependencies’ setup.py filles. Since these files aren’t idempotent, we can’t reliably cache the results (these scripts could return different sets of dependencies based on the environment or even the current time or anything else). This at least seems to be the reason all reproducible package managers are slow (by which I mean 30+ minutes to resolve dependency versions for a small non-toy project even if the resolved dependencies at their property versions are already cache to disk).
I think you and the first poster are talking about different things. They are talking about readability of the cpython compiler and you are talking about readability of the codebase.
Python has historically refused to implement many optimizations because they reduce compiler readability.
> Python's entire value proposition is that it is "executable pseudocode" (...) so somebody like a scientist (...)
LOL'd hard at that.
What kind of scientist would write "import numpy as np" on their pseudocode? Or multiply matrices with the "@" operator? From the point of view of a scientist, Matlab/Octave, or even Fortran, is executable pseudocode. Numerical stuff in Python seems like an ugly kludge.
As a scientist who had to use MATLAB up until about 2013 because that's what everyone else used, it was such a relief to move to Python. It's true that you can implement a linear algebra routine in a couple fewer characters in MATLAB, but unless that's literally all you're doing, Python is much nicer to work with. The data structures in MATLAB are just a nightmare for general purpose programming, which makes things like loading and parsing data -- things that scientists often need to do -- just terrible. The fact that the notion of the "matrix" (i.e. a 2D array rather than a general ND array) is so deeply baked into everything is a huge headache (a scalar is a 1x1 matrix in MATLAB!). I'm also surprised anyone is particularly bothered by the @ operator. The asterisk for matrix multiplication seems roughly equally unheard of in nicely typeset / handwritten math (you would just write e.g. Ab for a matrix-vector multiplication with no inset operator).
I shudder when I think of my Matlab days, but i’ve programmed long enough in enough other languages to know that we have better options than either Matlab or Numpy/Pandas with respect to API design. We can also have much better performance than Python with many other languages.
I don't know why you got downvoted, but I agree that matrix manipulation in numpy look quite foreign to to somebody new to python/numpy. I had heard this "complain" from several colleagues (mostly former matlab-users). Several colleagues have migrated to Julia, which does a quite good job in producing easy readable numerical code. For instance, the dot-product can be written with the Unicode dot symbol (typed as \dot<tab>)
> matrix manipulation in numpy look quite foreign to to somebody new to python/numpy
Not only to newcomers. I've been using python+numpy for more than 10 years, nearly every day, and it despairs me as much as the first day, if not more. I relish the few moments that I get to write numeric algorithms in any other language.
Julia's startup time is not short, but it's getting significantly shorter with every release. Right now, the startup overhead is only 0.13 seconds for me.
Some packages still take a while to load, e.g. for me to load the plotting pacakge and produce the first plot takes about 8 seconds (all subsequent plots in that session are fast). This is down from like 30 seconds last year.
One can also bundle heft packages like Plots.jl into their julia system image so they don't have to recompile all that machinery every time they restart julia.
The startup performance issue is just a regular difficult issue, as far as I'm aware. I don't think it has much to do with the goal of "being readable pseudocode" or something. It's getting better with time too.
The "readable pseudocode" kind of code is exactly the sort of code in Julia that you almost always expect to be compiled down to native code quite efficiently. The kind of pseudocode I usually see is either straightforward loops iterating over something where the compiler can infer all the necessary information, or calls to library functions where somebody else has already made sure it's good. I use this a lot in my own code, and, like I said, I don't think there is a tradeoff.
So in my mind, the main things you would need to compete with npm would be:
1. near-universal adoption
2. "just works" experience: i.e. I can clone any random git repo, and run `npm start|build|whatever` and it's always going to work, without having to know anything about my environment or fiddling with configurations.
In most npm projects, you don't depend on C extensions. For python projects that don't depend on C extensions, you already have your "just works" experience with poetry.
If you really want a "just works" experience when dealing with C extensions, you should freeze your development environment one layer up (e.g. VMs, vagrant images, what have you) so that you can always successfully install and compile your C extensions because the underlying system is also kept under version control. And this is independent of whether or not you are working on a Poetry or an NPM based project.
It's not that hard. I suspect that you're just used to simpler conditions.
From my experience, with repos that have Poetry set up, I do get a "it just works" experience that I've previously been lacking when working with python. AFAIK there is no "scripts" section (for e.g. "poetry start" like "npm start") in the config file, or at least no one is using them. You'll have to take a quick peek into the README, but the same is true for npm projects, as the script verbs aren't really standardized.
You can use a "scripts" section in pyproject.toml (it's what poetry reads, and increasingly more Python tools are leveraging it, which is good), but in general each repository/project offers what it wants to offer as commands, nothing in comparison with the "standards in use" for npm and similar. Personally I have used it to give easy access to the "main thing I want to execute" for my own weird projects.
I'm pretty happy using Poetry, and agree with your initial point, partially with the second. There can be some edges with packages involving "anything" binary (to be fair, that is to be expected), and a big issue (for me) is locking the resolution of some libraries, like boto (since botocloud has hundreds of patch releases and the resolver can get pretty crazy unless you play a bit of manual bisection). But this only hits you on poetry lock or poetry add when developing, and only in some cases, I think it's a fair price to pay for a reproducible build.
> AFAIK there is no "scripts" section (for e.g. "poetry start" like "npm start") in the config file, or at least no one is using them.
There is a scripts section in pyproject.toml, which is leveraged by “poetry run”. This is different than for npm, though, since these aren’t dev-environment scripts but the executable scripts that are installed with the project
For the case of Python packages which are primarily exposing a command line script, alluded in the `start|build|whatever`, there is pipx https://github.com/pipxproject/pipx
It seems a lot faster than pipenv.
From the little I've tried, it seems a lot more sane - but I'm biased - I hated pipenv and never saw the point of it.
It used to be incredibly slow but these days I find it's not so bad. It even has parallel pip installs now which while occasionally triggering some issues in projects with over complicated setup.pys is usually a lot faster than a good old pip install -r requirements.txt.
So have you considered the possibility that it may not be the fault of these package managers, but that there may be a different problem underneath it all that a package manager cannot fix for you?
It's a very common misconception that readability is a tradeoff against performance.
The true underpinnings of readability is language malleability, nothing more and nothing less. The whole "dynamic languages are automatically more readable" impression is a misleading consequence of the much more general claim "Malleable languages are more readable", with a malleability = dynamicism substitution.
Malleability is key because readability is really just languages being as close as possible to the problems they are used to solve. This means one of two things
1)Either the language come preequipped with the concepts and semantics of the problem domain built into their fabric, this is the DSL approach (this will always fail if you aspire to be a "general purpose language", you simply can't hope to match the sheer number of contexts people want to use your language in. One radical conclusion is to abandon the "general purpose language" myth absolutely, and just make all application development language-building, and focus on building the tools to make language building easy.),
2)Or you make the language malleable and stretchy enough that a tiny handful of the rare programmer-domainExperts breed can construct the whole domain inside the langauge, _with_ _the_ _language_ (no transpiler, preprocessor, etc..., this would require the rare programmer-expert to be an even rarer programmer-expert-languageHacker), then cover up all the low level machinery with syntactic elements that mirror the domain vocabulary.
That's it. Any langauge that allows you to do the above is a malleable language. Any malleable language is a candidate for being a readable langauge (and a horribly unreadable langauge, if you gave in to irresponsible abstractions). Dynamic languages are malleable because they give you extremely powerful hooks into their semantics, they are very... well, dynamic. The details differ, the two poster childs are python and ruby, they have a grab bag of features that range from operator overloading in python, free form syntax in ruby, and extreme dynamic dispatch and resolution rules in both. The last feature is a common theme in all dynamic languages, and it happens to be a performance killer.
But you absolutely don't have to be dynamic to be malleable, lisps have been doing it since forever, and though lisps are traditionally dynamic, it's macros that make them stretchy, not dynamicism. And there are loads of other mechanims that can make a language malleable without making it unpredictable. It can be as simple as scala allowing unicode in it's source, and as complicated as haskell making lazy evaluation the default and now control structures are just a library. Any abstraction with a "meta" flavor, with hooks extending into the language environment and doing various things depending on how you wield them, makes the language more malleable.
Malleablity is allowing the language's source to take different shapes and semantics according to the whimes of the programmer, performance comes from massaging the source till it fits comfortably into machine semantics without much runtime shenanigans. They are absolutely not in tension, making the code do everything at runtime is just the easy way out.
So much stuff just from the readme would introduce breaking changes to the Python ecosystem.
Part of the reason Instagram can get away with this is they likely have almost complete control of their dependencies and the like. But the changes they decide to make would not just work for everything.
That being said I think we could get a lot of neat PEPs from this.
EDIT: this is just my opinion , but in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff (and no black box hand waving for perf things).
> So much stuff just from the readme would introduce breaking changes to the Python ecosystem.
Being compatible with the rest of the Python ecosystem is the main reason why Cinder is built on top of CPython. Although yes, some features are indeed very experimental.
> in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff
Ah, but that introduces a separate compilation step, which may not be tolerable in every situation.
Why would developers have to interact with a mypyc step any more than the pyc step? Why is “developers might have to interact with it” some kind of non-starter, as though having a compile phase is a worse evil than a hyper-slow language?
FWIW, I think we could probably buy ourselves a lot of latitude to optimize CPython by designating a much smaller API surface (like h.py) and then optimizations largely won’t have to worry about breaking compatibility with C-extensions (which seems to be the biggest reason CPython is unoptimized).
But in general I’ve lost faith in the maintainers’ leadership to drive through this kind of change (or similarly, to fix package management), so I’ve moved on to greener pastures (Go for the most part, with some Rust here and there) and everything is just so easy nowadays compared to my ~15 years as a Python developer.
> Why is “developers might have to interact with it” some kind of non-starter, as though having a compile phase is a worse evil than a hyper-slow language?
For big monoliths (like ours at IG), the server start-up can take more than 10sec, which is already super high for a "edit -> refresh" workflow. Introducing a Cython like compilation step is really a major drawback for every single developer.
For smaller projects, Cython works extremely well (and we do use it for places where we need to interface with C/C++).
> For big monoliths (like ours at IG), the server start-up can take more than 10sec, which is already super high for a "edit -> refresh" workflow. Introducing a Cython like compilation step is really a major drawback for every single developer.
So we weren’t talking about Cython specifically, but something Cython-like, i.e., we’re not talking about Cython’s special syntax but rather ordinary Python. This is important because it means that dev builds execute against CPython directly (i.e., your code begins executing immediately) while production builds use our hypothetical AOT compiler.
According to the pypy people, the type annotations also don't provide the kind of information the JIT wants. The JIT cares about things like "this integer fits into a machine word" and "this parameter can be None but is almost always not None" so on. The type system doesn't concern itself with that.
It’s definitely true that there’s stuff a JIT would love to know that the type system can’t tell you. But that doesn’t mean there isn’t useful information available in type annotations. In particular it is possible to speed up attribute access and method/function calls a lot when target types are statically known.
Our approach to “int fits in a machine word” in Static Python is to require machine ints to be explicitly annotated as such, and then you opt in to limited size ints with overflow etc, in exchange for getting really fast arithmetic.
I mean if you care about perf you can do it in one step. It’s not that huge of an ask, and engineering wise “compile some typed code” is a hell of a lot more straightforward than “try to guess what I should speculatively compile at runtime”
There was a piece by v8 devs a while back where they ended up turning down JIT aggressiveness cuz it ended up making normal web pages slower (except for like Google Docs).
This is just me backseat-BDFLing “why do the hard thing when we could do the easy thing” tho
Typing a large legacy codebase written in a highly dynamic language is not as easy. Instagram is pretty large.
I have seen a very large codebase in a dynamic language at [redacted] that has been mostly converted to use a sophisticated type system. Certain core things have not been converted so far, though, but just marked with a lot of "proceed with caution" and "TODO" red flags. Making them typesafe in any conceivable way would break compatibility for large subsystems.
Sometimes a rewrite is the only way out of such a situation, but very few can afford it.
> Typing a large legacy codebase written in a highly dynamic language is not as easy.
> very few can afford it.
Starting a project in a highly dynamic language is a conscious choice with well-known properties. The second part is interesting, however. Essentially you are saying that gains from development velocity do not offset losses from technical debt, which is a strange observation. I would say it sort of hints at time to MVP being irrelevant business metric (startup runway/survivability aside) at best. This is contrary to typical startup truths.
At a certain scale, they do. Case in point: Twitter. They went through a painful transition but they did not have much choice.
But sometimes they don't. Case in point: YouTube, Instagram. Both of them are slowly migrating more and more Python code to different languages, but the amount of Python is still large, and AFAIK neither has plans to eschew it completely.
Time to MVP is a very relevant business metric. But the same thing that speeds you up in the beginning slows you down later on. If you architect your code past MVP to help replace the implementations of critical paths, it may help you down the line. (Micro)service architecture is one way to do that.
With Static Python we use type annotations in compilation, and we require them to be correct (they are runtime checked at boundaries with non-Static code and throw TypeError if wrong types are passed in.)
We haven’t gone the hidden classes route so far because it’s simpler to just look at attributes assigned in __init__ and annotated on the class and lock those into slots. If you’re writing typed Python you probably don’t do a lot of tacking extra ad hoc attributes onto instances, since type checkers don’t like that either.
> Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better.
I don’t know here you are picking this from. What’s hurting performance is not readability, it is the flexibility of what you can do (hence: strict modules/static python in Cinder) which means it is really hard to perform optimizations as no assumptions hold unless you turn down the more permissive language features.
Another one is backwards compatibility, and I might get some flak from people still complaining about it more than 10 years after python 3.0, but things are still 99.9% compatible with decades-old code in python 3.9, and you can’t throw that away.
Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages, and especially compared to the number of businesses using it. There are quite few people working on performance, but not many of them appear to be doing it in the open and with the goal of upstreaming it in a non-breaking manner (kudos to the instagram engineers, I guess).
> Finally, the numbers of people dedicated to improving CPython performance in an upstreamable fashion is ridiculously low compared to other languages
Yeah because Guido and the maintainers have proclaimed multiple times that they prefer a simple implementation for teachability, over performance minded design. That’s where this entire complaint is coming from.
This is why I see Julia's uptake as positive, because if that isn't going to change the community aversion to these attempts and will rather keep writing C, nothing else will.
I was doing a Hacker Rank puzzle yesterday in Python and after I optimised it, it still timed out on one input. Looked at the chat, and saw a comment "just run it with PyPy" and naturally it then passed all parts.
It's easy to forget that CPython is not that optimised generally, but as soon as you need to deal with algorithms it is lacking often.
I think even that the fact there are so many C implementations in libraries shows that the lack of optimisation for the sake of clean compiler code manifests as a readability / complexity issues elsewhere.
> It's easy to forget that CPython is not that optimised generally,
"Not that optimised" is actually an understatement. In my experience, CPU-heavy code typically becomes 10-100x faster if you port it from Python to a language like C (without even trying to be clever or using SIMD assembly).
That would indeed be fantastic. V8 has had so much optimization whereas CPython has had very little of it, though Pypy deserves a mention here. They've been doing fantastic work, but I guess there's always the curse of the reference implementation being the popular one.
I disagree, it was the Nim syntax that kept me away from the language back in the Uni when I tried it out. If anything, C syntax is barely as easy to read.
I got it to work with the following but errored out on openssl so I couldn't install pip.
It builds fine with docker fedora 32. Thank you.
Platform
Fedora 32, Docker on Windows 10, Version 2004
Steps to Reproduce
Install docker from docker.com
docker run -t -i fedora:32 bash
git clone https://github.com/facebookincubator/cinder.git
yum install zlib-devel openssl-devel
./configure
make
make altinstall
This error is occurs when you do ./configure --enable-optimizations this version of python complies it either way.
>> Objects/accu.o
Parser/listnode.c: In function ‘list1node’:
Parser/listnode.c:66:1: error: ‘/cinder/Parser/listnode.gcda’ profile count data file not
found [-Werror=missing-profile]
66 | }
| ^
If you get this error it is because you used --enable-optimizations.
The parent proposed gating for warnings in the CI, not (by default) on the developer build.
I completely agree with this, as a dev I want to see warnings but I don't always want to have everything perfect from the start. The famous example here is -Wunused-arg. I DON'T want to fight this during development, when I often have prototyping stages where functions remain unimplemented intentionally. I very much want to avoid merging this into master though, so CI gating with -Werror it is.
Also "I'm trying to build your code 5 years later with a version of GCC which has a load of new warnings you'd never seen". Dealing with those is not my problem, I just want to run your tool.
> We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance.
Nice. Looking forward to seeing that happen over the next few months.
Was Cinder influenced by hhvm (Facebook's vm for php/hack)?
A project that maintains a list of different JIT implementations for programming languages and compares them would be a great way to see what are the different approaches to implementing JITs and which language features make it hard to implement performant JITs.
As an aside it is great that the Cinder team is specifically calling out that Cinder is not intended to be used outside of FB. Many people have been burned by lack of community around hhvm.
> A project that maintains a list of different JIT implementations for programming languages and compares them would be a great way to see what are the different approaches to implementing JITs and which language features make it hard to implement performant JITs.
SOM for example has many implementations with different approaches to compilation http://som-st.github.io.
I'm afraid there is very little documentation/text on modern production JITters. When I tried finding any text for my MSc I had little success. Does anyone have a suggestion about e.g. .NET 3-tier jitting or similar?
> When I tried finding any text for my MSc I had little success.
Yes you basically need to sit down with an expert to learn this stuff. It's famously under-documented and extremely hard to learn how it's done in practice on your own.
There is a lot of value in leveraging existing programming languages than rewriting in something else. My team rewrote large chunks of code from python -> Go and it wasn't a pleasant experience. We were able to justify it, since it _inadvertently_ reduced our infra cost.
But, if we could make python a lot faster at compiler level, it doesn't break the dev experience and lengthen dev on-boarding time. This helps the team productivity as a whole. I hope the CPython community is able to leverage few things from Cinder. This would benefit the entire python community.
1. The computations are distributed enough that there is no one hot loop
2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot
1. In distributed computing, there are usually minimal communication because nodes communication is costly, the bottleneck can still be the hot loop on each nodes.
2. When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go.
I didn't mean distributed computing, I meant distributed through the program. Not all programs actually have a hot loop.
> When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go.
That assumes proficiency in C, and it still complicates building / bundling / shipping a lot, the first bit of native code into a Python codebase is a huge investment and a significant step back in convenience in many ways.
> "If a call to an async function is immediately awaited, we immediately execute the called function up to its first await. If the called function reaches a return without needing to await, we will be able to return that value directly without ever even creating a coroutine object or deferring to the event loop. This is a significant (~5%) CPU optimization in our async-heavy workload."
Step 1: "Who needs Threads? Just async/await all the things!"
Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there!
> Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there!
Addressing your snark, I assume this optimization is targeted at places where you have a single "await" expression in the source that might call different functions at runtime, only some of which need to await.
Or there are just different code paths in the called function (eg a “cache hit” code path and a “cache miss” one), and only some of the code paths need to await.
Looking through the repo, one change of note that doesn't seem to be documented in the README [0] is that this has an optimization to reduce Copy on Writes in forked workers (under the Py_IMMORTAL_INSTANCES preprocessor guard) by not dirtying pages via refcount changes.
Here's the BPO issue [1] that tracks the upstreaming attempt.
It’s also likely that this change, which is a big win for our prefork production environment, drags down our benchmark results. It makes every incref and decref more expensive.
> Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project
I'm quite surprised by that. I understand that FB makes tons of money so I can't believe they can't have the capacity... Maybe they don't have the will (which I perfectly understand, it's just the wording)
> We don't have the capacity to support Cinder as an independent open-source project
Our team is smaller than you might think! You also left off the last half of that sentence, which is quite important: "nor any desire for it to become an alternative to CPython."
We'd love to see the ideas and implementations in Cinder end up in upstream CPython in some form or another. It'll make our lives easier in that we won't have to maintain as much forked code, and hopefully everyone else will benefit from a faster CPython.
Instagram were heavily using Python (and Django) when acquired by Facebook, whereas Facebook proper were a PHP shop.
The most interesting aspect really is that both companies have run in to the limits of performance with dynamically typed languages and had to implement their own language runtimes, AOT compilers, and JIT's.
Do we have proof that they had performance problems? At least in Python you can write a C extension for a problematic code path which would probably take much less time.
I think what they are looking for is to reduce the amount of machines they need. Which in the end is more money in their pockets. Making the website faster for the end user is a welcome side effect.
There are articles like https://instagram-engineering.com/dismissing-python-garbage-... which seem to confirm some problems. One of the people involved with this Cinder project was an original author of https://github.com/microsoft/Pyjion (it's now being revamped by a new dev.) Dino had deep .NET/CLR implementation experience and had been the lead for IronPython which I think spawned the interest in perf improvements through Cinder. I hope this effort is more successful for him.
I love how Facebook and Instagram never went the route of "full rewrite" for their apps as they scaled.
I my experience "Language X is slow and we could save Y switching to Z" is always a false promise. You can pick parts of the system that are costing a lot and port them to other language/frameworks to capture the bulk of savings while keeping your developers happy working on familiar code. Or if you'e big enough like Facebook, you can go see why X is slow and if it is possible to improve at the compiler level. Never disrupt the developers flow (Twitter did this with their Scala craze back in the day)
If Twitter's port to the JVM didn't result in 10x fewer servers with 10x lower latency vs an identical Ruby implementation we wouldn't have done it. Sadly not much progress has been made on the RubyVM though Twitter did try with improvements to GC and other changes. We even tried JRuby as part of the evaluation.
Wasn't there a time when jruby was faster than the standard ruby implementation? On one hand, that's absurd, but on the other, a lot of effort goes into Hotspot, and you'll get those benefits for free.
Not really, it can turn out good from the JVM stuff for specific cases here and there, but it's mostly interesting for not having a GVL, thus enabling full parallelism for threads.
Nice. 10 years later but glad that Ruby programmers might have an option that lets them program the way they want without having to choose a different language / platform for performance.
It is still no where near the level that Twitter would have stayed with Ruby though. Although I believe TruffleRuby being 3x faster than current CRuby is only the beginning. JVM got tons of improvement in the past 10 years as well it is uncanny.
Although Dr Chris Seaton mentioned it is not his goal to have Shopify Rails running on top of TurffleRuby. Not sure if he meant it was short term or long term. I think Dr Stefan Marr will be joining him soon, and Dr Maxime Chevalier-Boisvert is leading a team working on YJIT in Shopify. ( Did Shopify just hire all known Dr. working on Ruby performance )
I'm not sure many companies have the luxury of doing what FB did. I can't imagine a world where choosing to rewrite your app, or choosing to rewrite the entire language, where rewriting the entire language would be the cheaper solution.
Twitter did this with their Scala craze, but Twitter struggled to scale it's Ruby app. Twitter didn't have nearly as deep pockets to write a performant alternative Ruby VM.
> keeping your developers happy working on familiar code
That turns pretty quickly into "developers being unhappy that they have to maintain legacy code written for the old language/platform," though, especially with any sort of employee turnover.
It doesn't have to be this way. If you're going to go all-in on the monolith approach like FB and Instagram did for their product code (at least when I was there), you need to have a central team whose entire job is scaling the codebase. This means actively refactoring other teams' legacy code, making sure that tests are fast, etc.
Yeah I have seen this successfully done at a smallish (200 person) company. There was a "platform" team which was essentially an "un-suck the monolith team".
Refactor slowly over time, turn a ball of mud into cleanly defined domain boundaries, parallelize the tests, etc. etc.
Idk, I worked at IG for a year and wrote a lot of backend code in the Python webserver stack. I believe they would benefit massively by aggressively migrating services to the FB Hack stack.
A ton of work goes into making sure the Python code isn't slow, and there are complicated C++ services built as workarounds for the slowness of python.
I disagree, this logic doesn't work always unless you have investor money lying around. For a bootstrapped business, it could mean living or dying. As an example, a client of mine was paying close to $5000 per month in server costs simply because of the scale of traffic they had on their site.
By re-writing Wordpress in Elixir, I was able to bring their costs down to $1000 odd per month. It's actually cheaper than what any of their competitors are paying for, in servers as well.
This $4000 in savings actually allowed them to hire an additional full time developer to maintain their site.
So, your logic only works for certain use cases, not all.
Saying that you disagree and saying at the same time that you rewrote it in Elixir makes me think that wasn't a good use of the money. They will have problems hiring for it and it isn't even performant compared to other languages you could have written into if you really wanted to save money (i.e: golang).
Of course I don't have all the context, but, I cannot imagine any circumstance when rewriting some server side logic in elixir would be useful from a PoV of cost savings..
How do you know it wasn't a good use of the money? The client's site has been up for a year since then without any issues for 1/4th of the cost. Their investment in this stack wasn't much. Hiring wasn't a BIG problem, but it did take time. We used that savings to find a dev to maintain the site.
By RAW numbers, GoLang or something else maybe faster, for sure, but it will definitely extend the time to develop something like Wordpress because its frameworks I've tried (Eg. Gorilla) need more effort compared to using something like Phoenix that I use. Plus, a lot of devs were already onto Ruby, it's easier to onboard them into Elixir than GoLang.
> Of course I don't have all the context, but, I cannot imagine any circumstance when rewriting some server side logic in elixir would be useful from a PoV of cost savings..
I think you may not have worked with a high traffic site, just by swapping languages alone you get insane cost savings for that scale because it quickly adds up. Even if your new language is just 0.1 second faster per user, for a million users, it adds up really fast. As I mentioned, just by this switch alone we brought down the cost to 1/4th.
I believe GoLang or something else may have drastically reduced further, but it has the same downsides of Elixir (Harder to find talent for, learning curve). I'm a pure Elixir consultant, so I chose it. My client is more than happy with this outcome.
Yes, I kind of agree with everything you said actually! Apart from Elixir from a cost savings on infrastructure PoV, if that's how you wanted to frame it (since the context of the HN submission is about changing languages for better performance or not, and Elixir not being particularly fast).
I'm obviously not going to reveal actual numbers, but it's safe to they spent less than what they would have spent being on Wordpress for the remainder of the year.
Time to completion is 3 months. No new "feature" development was blocked. Remember - this is a content website (Wordpress is mostly used for content AFAIK and less for building features).
The moment it was done, we just switched nameservers. It had no impact on client's workflow or day to day life.
If it was a content website on Wordpress, couldn't Cloudflare's $5/mo APO have done the job? It static caches the site at the edge. You'd get almost a 100% hit rate unless there are strange requirements.
Good suggestion, but if your content site is a news site, then homepage and certain category pages require by the second updates. Cloudflare unfortunately can't help and all the requests are directly hitting your database at that point. That forces you to get an expensive MySQL instance to sustain that kind of traffic.
Cloudflare will automatically purge the cache when you update or publish an article. Also from my understanding of APO, the cache will get distributed to all their edge locations without each one needing to hit the origin server. So if you update an article it would only get fetched a few times max until it gets updated again. That seems pretty good. Also as far as MySQL, you could install a Redis cache in front. Personally we use Varnish and Memcached as additional layers in front of Cloudflare (it works out of the box with Cloudways)
Safe to say they would've gone out of business if they kept burning that kind of money for the remainder of the year. This was around covid era, they didn't have a solid revenue stream from the traffic they got.
You can't cache everything on a news website, especially the homepage. The homepage requires constant updates. For rest of the pages, I agree and I do use caching.
This is some of the original thinking around StackOverflow, if I remember correctly from the podcasts. Jeff was a C# dev and using what was familiar, but he also commented on how you could get good performance out of some crappy code and easily refactor later.
Their whole culture elevated performance and they handled a lot of traffic with relatively small amounts of hardware that they documented. I've got some familiarity with a Rails app where the devs seemed astonished at some of the details in their blogs about running SO.
One factor for Twitter was that Ruby was simply orders of magnitude worse for Twitter usage than I think Php or Python was for Facebook or instagram. Or at least one never heard of these providing horror stories where Twitter definitely had horror stories. In one of the few times I've been "close to the source", I heard Twitter's head of operations complaining the electricity had become their primary expense from starting X many Ruby instances per second. It was one instance per connection or something close, because Ruby was unstable and because Ruby never gave memory back to the system.
This is something that people have known for a long time: a lot of money has been spent on making compilers make fast code. For many intents and purposes we know what kind of reasoning we want to be able to do to produce fast code. Strict modules is probably one of the sinples yet efficient optimizations you can do: being able to reason about where and what something is without having to do an expensive lookup every time it is used is brain dead simple to understand.
Yet many of the languages that evolved as "scripting languages" threw much of that knowledge out. In the middle of the 90s we knew how to make fast dynamic languages. Maybe not as crazy dynamic as something like ruby, but not far off. Or at least, they had the sense to codify it as a separate thing (CLOS in common lisp comes to mind).
Something like SBCL is more dynamic than you need(TM), yet produces code that is often at least an order of magnitude faster than python.
Yes, Static Python especially relies heavily on strict modules, since they enable us to perform module-local analysis, which enables some cool optimizations.
"Language X is slow and we could save Y switching to Z" is always a false promise.
The thing that keeps software interesting for me is that there are almost no absolute rules. So I'll agree that complete rewrites are usually a mistake and performance problems are often not in the language.
But I can't agree that rewrites are always a mistake. esbuild is a recent good example of how much difference switching to a faster language can make.
Did Twitter actually contribute to Scala core though? They definitely created and contributed to various Scala libraries but they didn't fork/augment the Scala compiler like Facebook did with HipHop.
Not sure I can agree with that. Sinking resources in to re-engineering a language infrastructure, mostly because the language / infra is not actually a good choice for the target problem is an approach only very large engineering teams can tackle. As has been already been started (further down the thread). Breaking down your code / monolith into manageable components and re targeting for performance in the key spots is almost always the right approach. Unless of course you have the engineering resources and hubris of someone like FB
Rewriting in full for the sake of performance gain is indeed a questionable proposal. Rewriting hot spots being the more sensible approach.
A better reason for rewriting however is to migrate to a more maintainable language with focus on readability from one which focus on expressiveness (and hence favored for rapid development) as the project matures.
I think while performance is not a proper justification, if you get other stuff in the bargain it can be really worth it. Rust, for example - the justification to switch is usually not just speed but safety and the safety enabling more optimizations, parallelism etc. If all you care about is speed you stay in C/C++.
Sometimes the choice is also made when you already have to pay a transitional cost - at a previous employer we had to rewrite a bunch of code to move from an old version of PHP to a newer one, and a strong argument was made that at that point we should just adopt a better language since we already had to pay that cost. I think eventually a lot of stuff transitioned to Haskell as part of that process, even if other stuff stayed in PHP.
It’s worth noting that we’ve never targeted perf on any of these benchmarks, so these are just the results that fell out from optimizing for our production workload, which is a totally different beast.
Most of the hits are due to the default “JIT everything” behavior which is really bad if there are a lot of rarely called functions, but would be easy to fix. This is discussed in the README.
This is cool, but at their scale I wonder if they wouldn't be better off rewriting most of their codebase, or at least a performance-critical subset, in something like Rust or Go.
It may seem like a lot of work, but making Python faster isn't exactly trivial either.
Also, I love Python, but using it for huge projects like this sounds dubious at best. I think the benefits of static typing really show when projects grow large.
We’ve chatted with kmod a few times and let him know we were open sourcing Cinder. Hopefully the projects can learn from each other. As _carljm mentioned, Pyston was restarted way after Cinder was in production and we had already implemented a significant amount of the core JIT functionality.
Cinder is a fork of CPython, and maintains compatibility with C extensions written for CPython. Which includes a lot of important/popular third-party packages these days.
Static Python is a bit different in that it’s not a subset language. It makes a few semantic changes, most notably that classes are slotified, so you can’t tack random extra attributes into instances. If you’re writing typed Python you probably don’t do much of this anyway since all type checkers will complain about it. But the approach is “normal dynamic Python with optimizations where allowed by types,” not subset.
Unlike RPython, Static Python in cinder is not really a subset of Python, it can compile everything (although it will throw compile time errors if it sees mismatched types). If it cannot determine type information, it just assumes the type could be anything, and falls back to slower CPython behavior.
In many ways nice that they open their performance work, some of which could be upstreamed.
In some ways a bit cheeky that they took the 'dump it and see' approach rather than offering to work to upstream it, since it's kinda outsourcing the work of maintaining the performance improvements to the Python core devs rather than offering to put some of Facebook's considerable resources towards doing it themselves.
What more could we expect from them to do? They are a for-profit company. Opensourcing their entire interpreter stack with years of proprietary development is a bold move. It shows that they care about open source and want to give back to the community.
They could've kept these developments behind closed doors and nobody would mind it.
They also offered to provide support if anyone wanted to contact them. That is top-class behaviour from instagram, I personally couldn't expect more.
Not saying what they did was not good; it's certainly better than keeping things closed, but to answer these two parts:
> What more could we expect from them to do?
> That is top-class behaviour from instagram
To be considered top-class, I would expect them to go the extra mile and contribute to upstream. As many other companies have done over the past and continue to do.
We have contributed quite a few parts of cinder upstream already and will continue to do so. I’m not sure where you got the idea that this is just a dump and we expect core devs to do all the work of upstreaming.
I'm very happy to hear that _carljm. I was just replying to my GP's "they took the 'dump it and see' approach" and my Parent's "What more could we expect from them to do?"
If you're going that mile, then my hat is off to you and the team, that's how it's supposed to be done! :)
"Open it and see if there is interest" is the first step in "offer to work to upstream it." Not sure where you got the idea that it's only the former and not the latter.
> Python can be ahead in the performance race. We just need to get real.
Completely agree, and Instagram did it, and they have been reaping the benefits for it. Now they've open-sourced the runtime, so we can all benefit now too.
in the source there are quite a lot of references to a GIL (global interpreter lock). Does Cinder offer any improvements in terms of locking/GIL ? I didn't see any mention in the documentation. https://github.com/facebookincubator/cinder/search?q=gil
There'll be a talk on Cinder at (virtual) PyCon in a couple weeks, there are detailed benchmark numbers in the talk. We've talked about getting them into the repo too, might happen.
We tried PyPy without much success (and we really tried, six month project just to get it to run our codebase.) Our workload is very heavy on the CPython C API, which makes life difficult for PyPy or any other alternative Python implementation. PyPy is great for a lot of uses, just not a great fit for us.
It was a few years ago and I didn’t work on it directly, so my knowledge is a bit fuzzy, and I think the person who did work on it is no longer at the company. As best I recall the slog to get it working was mostly about C extensions, but I don’t know details. And then once it was working the memory use was worse and it didn’t show perf improvements. This is speculation but I would guess the design of the pypy jit isn’t a great fit for a prefork workload, where the hot paths don’t get hot until postfork, but it’s pretty inefficient to do identical jit compilation in every worker instead of once in the parent process.
This is meaningful for some big web workloads. By cranking up your Python codebase performance you can avoid expensive rewrites in harder to maintain languages.
I think this section of the readme speaks to that:
Is this supported?
Short answer: no.
We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance.
Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project, nor any desire for it to become an alternative to CPython.
Facebook is basically just open-sourcing their changes to CPython, in order to make it easier to work with core CPython developers. For a company of Facebook's size, migrating to PyPy would probably be have been a much bigger endeavor than simply forking CPython. Since the goal of this open-sourcing is to improve CPython rather than to attract people to a competitor, I would interpret this work as "Facebook investing in CPython" rather than "Facebook investing in a CPython alternative".
What a ridiculous conclusion you came to based on a single line from that page. The very next line says:
> We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance.
> Our goal in making this code available is a unified faster CPython.
Facebook is a Python Software Foundation sponsor[1], and they sponsor every PyCon. I don't know why you got the idea that they don't contribute back.
Yeah but also... “If you want to upstream it, take it” it's not really something where ehi forked is working hard to send it back upstream patch by patch...
We've already upstreamed a lot of changes, including one that makes all coroutines faster in Python 3.10. The big-ticket items in Cinder would be big changes for CPython and need discussion about whether CPython even wants them. You don't just slap a C++ JIT into a bugs.python.org ticket. Opening Cinder is the first step in the conversation, and we're here for that work too. It's not just "if you want it, take it."
The PSF has not really sponsored CPython so far, and PyCons are for self promotion.
Often in OSS foundations the entire sponsorship money goes to people who don't deserve it.
Please do not attach any meaning to corporate sponsorship of foundations unless you can point to a specific project.
99% of CPython has been written by individuals without any compensation who don't get any credit. When a corporation gives money to non-productive bureaucrats everyone thinks they are helping open source. It is very sad.
Instagram upstreams a lot of changes to cpython, and most of the IG backend runs on services outside of python/Django.
I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work. IG is saying "we built this stuff, we're making it open source". They are not asking for or expecting anything from the open source community or you, but you are asking for a lot (millions of dollars worth of work) from them.
Exactly - totally pathetic. Seriously - what are you contributing in terms of engineering budget? Nothing? Then stop bothering contributors to open source.
> I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work.
That depends on the license, though. It's not entitlement to think that someone that builds on something open must release it back if the license requires it. That's one of the main differences between the BSD license and some GPL variants.
Years ago, Facebook maintained a Python SDK for their API. One day, with no warning, Facebook announced they would no longer support it because they didn't have the resources, and the repo was removed from their Github org, which caused a huge headache. IIRC, the community settled around someones fork.
A few months later, Facebook was a major sponsor of PyCon and they set up a recruitment booth. "We'll take your developers, but we won't support your ecosystem." Really rubbed me the wrong way.
not trying to throw sympathy here for Facebook, but the team that owned that Python SDK was probably not at all related to the PyCon recruitment booth. Big organization, at least someone was trying to do the right thing
CPython explicitly doesn't want these "improvements" CPython has explicitly declined performance boosting pull requests if they add too much complexity to the code base. The two project have very different goals.
This seems unnecessarily cynical. The large block of caveats seems mainly designed set expectations for potential users about the level of (total lack of) support.
Using (and modifying!) a piece of software in accordance with its license is not appropriation, it's the point of open source! Users - even corporate users - are free to make changes to suit their needs and aren't required to seek anyone's permission do so. All changes do not need be contributed back to the parent project, especially if (as the sibling comment notes) the changes don't align with the goals and values of parent.
That this hacked-up internal fork is even seeing the light of day is a Good Thing.
Open source was never meant for "corporate users". The spirit of open source was to create free alternative to expensive software so that disadvantaged people could also participate in a digital revolution.
The whole movement was then appropriated by big corporations when they found out the developers will do the R&D work for free (so corporations don't have to pay salaries and taxes on the work done) and then if any of the projects become valuable they can just take it and use for their own purposes, make money and never pay original developers anything whatsoever. Of course some companies felt guilty and offered jobs to those developers - but when you compare how much money they made on the software vs what kind of salaries they pay, it's pure simple exploitation.
The whole open source movement currently is designed to exploit developers and is a mean for companies to avoid paying for labour.
We've been calling it Cinder internally for years, long before we thought about opening it up. If we wanted to make it a big branded thing, we'd work on finding a unique name for that, but we aren't trying to do that, we're just opening the code to make it easier to share work with upstream and other people working on CPython performance. We hope libcinder keeps the mindshare around the "cinder" name.
Too bad it's destined to be abandonware. Just think of all that money and engineering time that could have been spent working on CPython upstream, rather than a fork that'll die when the company gets bored with it.
Aside from faster CPython, I'd love to see a PIP that standardizes module naming convention & inheriting+extending existing modules, rather than completely reinventing the wheel and making a totally new module just to add a single feature that some other module lacked. That plus the new regex extensions and Python will be almost as useful as Perl. :-)
The readme does say that the goal here is to either get this work upstreamed to CPython or to inspire/inform improvements to it, not to replace/supplement it.
But it's probably not going to go upstream. And probably very little of it will go into other projects. Every time one of these forks gets dropped on the internet, nobody does anything with it, because it goes out of date shortly after it's dropped. Probably a couple specific optimizations will get picked up from it and put upstream. Versus if development had started and stayed in upstream, you wouldn't have to force people to pick through a huge patchset looking for things they personally want upstreamed.
We have multiple CPython core developers at Facebook, and the Cinder team (among others) are heavily involved in discussions around CPython's direction. There are many pieces of Cinder that would never get accepted upstream, or would take a decade of PEPs and incremental improvements to get there. Cinder is both a proof of concept that these ideas can work in CPython, and something that we can use today, rather than waiting a decade to realize these benefits from a pure open source perspective.
My experience with Django was that the larger the project got, the more I wanted to bypass it and just work directly with the database. Instead of mixing in python code in html. I just wanted to output quoted html in python. I got into weird issues with the parser. Like I wanted to output a dynamic style sheet. And it wasn't updating properly, probably because of nested includes.
Its also much nicer to reason about a function outputting a block of html code that represents some object like a toolbar. Than to have python code in the html doing doing for loops with html code inside.
Its probably because I needed a page to act differently if it was embedded or not, desktop or phone, and if the user was logged in or not. So in essence 2x2x2=8 different behaviours. Even things like links would need to change inside the page.
Easier, and shorter to write a function that changes the output. Then to battle the templating language, or write and support 8 different views.
Some of that logic (logged in, or exposing stuff based on RBAC) are usually pretty easy in templates of any flavor. And that's usually where I have to revisit for changes, so maintenance shouldn't be bad.
Sometimes I do common things like stored markdown->html by injecting filters into the template context. So {{ comment | markdown }}.
For completely different presentations, you can have separate templates returned by the same endpoint. But it sounds like that actually should just be a separate endpoint altogether. You could use template inheritance for code reuse, but you can also walk into over complicating that if those start diverging over time.
Handling screen variety is best done with CSS (and JS when you have to do something hacky). The HTML shouldn't need to change for that.
Even back to JSPs or early PHP, I learned the hard way to avoid sticking business logic in my "templates" or sticking HTML in my code. Either will work, but maintaining it sucks. And if it is a successful project and you need to change the design in year 3, it'll easily result in a nearly complete rewrite.
I know the cool thing to do now is even just expose the data via REST or GraphQL and do a SPA, but I don't do frontend work enough to get over that learning curve.
I don't get it... templates sound like what you want here. You want links to change based on some conditions? Why not use {{ generate_link|safe }} on one generic template?
Thats what I ended up doing most of the time. Still ended up with a lot of if then code in the html template. Making it more verbose than it would have been, had I just generated the html page inside a function.
You can often generate a structure such as nested dicts and lists of dicts to pass via the template context to achieve your desired layout. This lets you avoid complex logic via template tags but still have the benefits of the templating system.
Python has emphasized readability for the reference implementation vs. the practical benefits of better performance for everyone, and the community is really hurting for it.
Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better.
Python can be ahead in the performance race. We just need to get real.