I'd be extremely (and pleasantly!) surprised if this turns out to be real. It is very common for people to compile a subset of the language and achieve great speedups, only to discover it doesn't scale to the full generality of the language. That would be my guess for what's happening here.
Best of luck to the Nectar team, looking forward to seeing what you've got. If this works as you say it does, you'll have built something incredible!
Sure. Basically, a bunch of scripting languages (Ruby, PHP, Python, a few others) share the same properties, that make them hard to compile:
- they're not really specified anywhere
- they use `eval` and similar dynamic constructs
- they allow you to mutate your data in weird ways (in particular, you can change variables dynamically by knowing the name of the variable, aka "variable variables").
I looked at how to solve these problems in PHP by embedding the PHP interpreter into the compiler as well as into the compiled program. Then I looked at how to do alias analysis in PHP - alias analysis is a type of static analysis to figure out which names in the program point to the same value. This had been done before for a subset of PHP but not for the full generality of the language, in particular dealing with variable variables.
There's actually a not-too-terrible talk going over some of the challenges of compiling PHP, and how initial promising gains turn out to be very difficult (as OP said) to maintain as you try to encompass the full language. Some great examples of disappointments and challenges!
> - they allow you to mutate your data in weird ways (in particular, you can change variables dynamically by knowing the name of the variable, aka "variable variables").
There are a few ways to naively do this. The first and easiest I'd say is just to make a jump table of all the variables in scope. Take a hash of the string and access that. The hardest way is to optimize out this access. Check to see where all of the callers are and inline the function to make the string hard-baked in. After that you can simply replace $some_string() with string_contents(); and get 0 overhead.
> - they use `eval` and similar dynamic constructs
This is harder but eval is largely discouraged and mostly unused (I hope). It can still be supported by linking in the compiler at runtime and calling the compiler on the string passed. It's slow but would allow it to work. Alternatively you could also do the inlining optimization trick.
I'm not sure why you wrote this comment. Did you think that someone who did a PhD in this didn't consider this? The difference between writing this comment and actually trying it is why they give people PhDs.
Have a read of my thesis. I researched whether eval is used much (answer, yes). I looked at the challenge of inlining (you have to know what function is called - non-trivial). The naive jump-table thing you describe is basically impossible (or at least, it makes optimizing around it impossible). If you don't know the string value, all bets on all variables are off. You need to keep the value out of registers, you can't reorder operations, you're basically at square one.
I'd be extremely (and pleasantly!) surprised if this turns out to be real. It is very common for people to compile a subset of the language and achieve great speedups, only to discover it doesn't scale to the full generality of the language. That would be my guess for what's happening here.
Best of luck to the Nectar team, looking forward to seeing what you've got. If this works as you say it does, you'll have built something incredible!