What's the relationship between github/ruby and ruby/ruby? It looks like they've diverged quite a far ways away from each other, but that might just be an artifact of which branches GitHub uses when comparing the two.
Yes, Aman and Koichi work very closely, the main point of difference at the moment is the method cache patches, Koichi is working on getting something similar implemented in MRI
As if the Ruby Community has more then enough resources to improve on TWO Compiler / Interpreter at the same time. Isn't it better to work together then to reinvent the wheel?
Speaking of compiler, the JIT for Ruby development has been very quiet for months.
The Lua community has found that bytecode is actually slower to load than it is to generate from source:
The extra latency of loading the (larger) bytecode from disk/ssd/flash, exceeds the cpu time to lex/parse.
I don't think so. There are a few things that looks distinctly iffy in that respect on the surface, but they are resolved.
e.g. is "foo" a method call or an instance variable? You can't know in isolation, but it doesn't matter at parse time, as if it's part of a larger construct that is only valid as a method call, such as if there's an argument list after "foo", it is parsed as a method call. E.g:
foo = 1
foo(42)
will result in:
test.rb:4:in `<main>': undefined method `foo' for main:Object (NoMethodError)
I think all of the potential cases that might have otherwise made Ruby impossible to formally parse are resolved in similar ways.
Now, there are certainly layering violations. The aforementioned example of "foo" by itself can only be resolved by determining whether or not "foo" is in scope as a local variable at the point it is referenced, for example, but you can opt to defer the decision until after parsing.
ruby has a parse.y yacc grammar source file (linked to from the post I linked), and yes it can be parsed separately from being executed.
However, IIRC, no one has been able to re-implement parse.y; jruby and the other ruby re-implementations copied it and made what adaptations were necessary.
- you don't drop the disk cache; so you're loading from RAM (echo 3 > /proc/sys/vm/drop_caches)
- you don't have very complex code
- you timing includes starting/loading the intepreter itself.
Yes that is why I said the benchmark measures "just CPU impacts."
I don't think dropping cache alone would be sufficient to simulate loading of a large project -- I think the benchmark would also need to split the input across many files to recreate the seek time effects.
> you don't have very complex code
Given what I know about parsers, I highly doubt that the performance profile of parsing real code differs from my benchmark very much (ie. >30%). But I'm happy to be proven wrong on this, if anyone wants to try.
> you timing includes starting/loading the intepreter itself
That is why the benchmark makes the files pretty big, so the constant interpreter startup overhead is not a significant factor. Again, I don't think that you're going to be able to significantly change the shape of the results by adjusting for this.
Most of the time that is likely due to bundler/rubygems stuffing your load path full and causing thousands of unnecessary stat calls.
The actual time spent loading/parsing files is in most cases a tiny fraction of the startup time of any large project using rubygems and bundler.
I counted several hundred thousand unnecessary stat calls on the biggest app I have, and ended up with a ugly hack where we trimmed the load path around each set of require's to only paths needed by that specific gem.
What is the total amount of time you spent during those few times, although including the amount of time to learn what happened, and how many thousands of times larger is that amount than the sum total of the time saved by caching bytecode compilation every single time you loaded a pyc for every file you ever wrote?
The overhead is tiny, less than milliseconds for sane modules. It's a useless optimization, especially when it can be done at install time only, say, as opposed to for every module import, but even that is somewhat silly.
Without seeing any numbers, it doesn't mean much to me. I'm assuming someone much smarter than me has identified the benefit of bytecode caching and unless it really gets in my way, I see no need to do away with it.
Refactoring a module to a package (e.g. mymodule.py -> mymodule/__init__.py) can cause errors if you don't clear the bytecode, since it can pick up mymodule.pyc and fail to see the new files.
Not sure about the exact conditions when this occurs, but it's definitely happened to me when code is refactored and you pull the newest version in with git.
speculating: it allows you to still be fine with a slower parser. and this means you can theoretically make the parser smarter (like make it do some inference and catch some bugs at parse-time?) without worrying that much about performance. but then again, I can't imagine what kind of bugs could a parser catch for such a dynamic language like Ruby, so they're probably just doing it to shave some milliseconds from program-start-up time...
With huge libraries (like SpreeCommerce, for instance), startup time can be seconds or more even on recent hardware, and the vast majority of that code isn't changing between runs. So if every gem was cached, they could see radically faster times to load their test suites across their entire organization. It's well worth it even without a smarter parser.
No, Ruby behaves like APC by default; code is loaded and parsed once per process, and then the AST is reused from memory. You can do some stuff to hot reload from disk (ie, Rails' reloader), but you have to intentionally do it.
This would just improve the time to initially start a Ruby process.
PHP needs an opcache because its standard behaviour is to discard all state after each request - including all compiled bytecode - kind of emulating the CGI model.
Ruby web apps on the other hand tend to be run in a loop - the script never exits after a request, it just goes back to the top of the loop to accept the next request to serve.
It's nice because it completely decouples your app's startup time from request processing time, so you can do expensive setup stuff up-front without slowing down each request. The disadvantage to that is there's not much pressure to keep startup time low, since it's only happening once.
