(Whether I recommend it, not sure! I did it and then undid it, with suspicion that tests were taking longer due to, perhaps, worse caching of build artifacts.)
As you get into this, you’ll want to improve the tokenization to something less naive. (The naive approach is fine for the educational purpose of the article.)
I’ve made the mistake of simply splitting on spaces or punctuation, and eventually finding that will do the wrong thing.
Eventually I learned about Unicode text segmentation[0], which solves this pretty well for many languages. I believe the default Lucene tokenizer uses it. I implemented it for Go[1].
Yea agreed, as to my tokenization example. Back when I was doing search for a price comparison service. The model numbers of the products were sometimes entered as LGA775 or LGA-775 or LGA 77-5 or LGA—775 (unicode dash) or "LGA 775"
And that is just one example :)
I ended up by developing a suite of "search tests" so whenever we were doing tuning (daily basis) we had a set of automatic search tests of about 80 weird/prev-wrong search cases we could check in a blink of an eye. One could specify the top X result needed to include at least Y number of products in XYZ category or have "brand" as ABC etc..
One might say it was "test-driven-search-tuning"
What allows a firm to stay competitive if there are many apparently wasteful tasks being done? Should that firm not be superseded by competitors offering better efficiency (reflected in prices) or consumer benefit?
An industry that accommodates a lot of bullshit work is an industry with high transaction costs. The author hints at that at the end of the article, but leaves it an open question.
IMO The problem as it stands now is that when developing typescript you have a compiler running on your machine doing the typechecking. It will error out and simply not compile when you try to pass a string to a fn expecting a number.
What if there is a bug (or even a misconfiguration in your project)? Or typescript changes over time (and now there are different versions all needed to be supported)? Or what if the types from typescript (designed for programmers) are not always inline with V8 internal representation of objects?) Or what if people push typescript through V8 with compile time errors? (The compiler should have prevented this from happening - but you can pump whatever you want into V8, including things considered broken by typescript).
You only really get any improvement in runtime if your compiler guarantees anything (and bakes it into whatever it is compiling). As of now there are little guarantees with the javascript language (which is what comes out of your typescript compiler). AFAIK typescript is designed to help programmers find common bugs.
Not saying it's not possible, but more work is needed into bridging the two.
I quite disagree. Look at JVM performance, there’s been huge improvements over time and a Java program compiled 20 years ago will get the same improvements as one compiled today.
Not only that but you can expect to get better performance from the JIT than AOT because the JIT can perform optimisations which are impossible for the compiler.
The compiler can also perform optimizations that are impossible for a JIT because they're too expensive. JITs have to balance execution speed with compilation time. It's not so clear cut which approach in theoretically superior. I also don't know any languages where a JIT and an AOT compiler have gotten equivalent amounts of engineering hours so that we could have a fair comparison.
Graal can run in dynamic and aot compilation mode. If you feed profiles into aot from a test run you can get pretty close to dynamic compilation peak performance.
> you can expect to get better performance from the JIT than AOT
And you are likely to be disappointed. I think these claims have been made for the last 20 years, and Graal might be the first to deliver something general - all previous successes were very limited to small niches.
I like JITs and what they offer, but very consistently for me and almost everyone I know, in practice AOT works better (faster, consistent, no "heuristic changed in JIT version and now it's slow" surprises).
> Not only that but you can expect to get better performance from the JIT than AOT because the JIT can perform optimisations which are impossible for the compiler.
I can’t help myself, but the JVM performs at par with managed AOT languages (Go, Haskell, OCaml, etc) and strictly slower than the unmanaged AOT languages (C, C++, Rust). The JVM does outperform various AOT Java compilers, but that’s probably an artifact of the decades of optimizations that went into the JVM; the same investment in an AOT compiler would close the gap to within the margin of error. Anyway, sorry for my compulsive nitpicking; hopefully it was interesting.
JVM vs C++ depends very much on the code shapes and what the code is doing. C++ that's very heavy on virtual function calls can be faster written as Java. On the other hand if you use a lot of SIMD intrinsics and things like that, C++ can be a lot faster.
W.R.T. AOT vs JIT, as others are pointing out, Graal is a compiler that can execute in both modes, i.e. it's a comparable compiler. JITC is about a 20% win for relatively static languages like Java and can be massively larger (like 4000%+) for other languages like JavaScript or Ruby. In fact the nature of very dynamic languages like Ruby or Python or JavaScript mean there's little point trying to AOT compile them at all because there'd be almost no optimisations you could safely do ahead of time.
You make it sound like Java is neck-and-neck with C++ in general. There are definitely cases where naive Java is faster than naive C++, but those cases are infrequent. Java is generally in the ballpark of 1/2 C++ speed alongside Go and C# in the general case.
> W.R.T. AOT vs JIT, as others are pointing out, Graal is a compiler that can execute in both modes, i.e. it's a comparable compiler. JITC is about a 20% win for relatively static languages like Java and can be massively larger (like 4000%+) for other languages like JavaScript or Ruby. In fact the nature of very dynamic languages like Ruby or Python or JavaScript mean there's little point trying to AOT compile them at all because there'd be almost no optimisations you could safely do ahead of time.
It seems like you mistook my point for "AOT makes things fast and should be used everywhere" or "JIT is fundamentally slower than AOT"; my only point was that the JVM specifically isn't faster than AOT languages nor is it faster than AOT Java because of JIT specifically (which was how I interpreted jahewson's comment above). It sounds like I agree with you--Graal seems really promising and JIT is pretty great, especially for dynamic languages. :)
Well, C++ is such a flexible language that it's hard to say what general C++ code looks like. I'd say I'd expect Java to either match or even beat C++ for general "business code" which is pretty vague but is typified by lots of layers of abstraction, lots of data transformation, hash maps, string manipulation etc. Modern compilers like Graal are very good at analyzing and removing abstraction overhead. I'd expect C++ to stomp Java for any sort of numerics work, media codecs, 3D engines, things like that.
my only point was that the JVM specifically isn't faster than AOT languages nor is it faster than AOT Java because of JIT specifically
Hmm, but is there such a thing as an AOT language? You can interpret or JIT compile C and you can interpret, JIT or AOT compile Java too.
I think it's clearly the case that JITC Java is faster than AOT Java. It was maybe unclear for a long time but Graal and SubstrateVM let us compare now. You pay a big performance hit to use a VM-less AOT Java compile. Java is sort of a half-way house between a really dynamic language and a really static language.... it's got dynamic aspects and also static aspects.
> Hmm, but is there such a thing as an AOT language? You can interpret or JIT compile C and you can interpret, JIT or AOT compile Java too.
You're right that my language was imprecise, but my point stands. JVM does't make Java faster than popular AOT implementations of (for example) Rust or C++ or C, and JVM is almost certainly faster than AOT Java implementations because of decades of optimizations, not because JIT is inherently better than AOT.
> I think it's clearly the case that JITC Java is faster than AOT Java. It was maybe unclear for a long time but Graal and SubstrateVM let us compare now.
Graal is shaping up to be a real game changer, but it's unproven and it's exceptional among JIT implementations. If Graal is all that it's cracked up to be, I wouldn't be surprised if there is a time when all popular JIT implementations perform like Graal, but until such a time, JIT alone doesn't have any clear performance advantages over AOT in general.
> because the JIT can perform optimisations which are impossible for the compiler
Is there an example of at least one such optimization? As JIT is not magic, it also compiles things ahead of time, but a little bit ahead of time and it is very constrained on how much time and resources it can waste. It also can't afford to slow down interpreter too much, can't do excessive profiling, can't do big changes to the code that touch half of all the functions and data structures for example, etc. While AOT can do that and I can't think of anything JIT does that AOT can't do.
One such example is dynamic re-compilation in Java, with e.g. inlining or uninlining based on recent profiling. This would handle the case where a certain (large-ish) function is on a branch that's often taken for some stable amount of time (where inlining gets rid of method call overhead), then often not taken for other periods of time (where un-inlining gets rid of the instruction cache bloat).
Artificial example:
while (true) {
if (timeOfDay < NOON) {
computePi(); //should be inlined before noon, function call after noon
} else {
computeE(); //inline after noon, function call before
}
}
@JIT vs. AOT: i used to think that, too. but graal came out and got 90% of the JIT speed right from the start with a fraction of the time hotspot had for optimization (and of course, with better startup times).
Yes, it seems Graal might finally deliver on the "JIT is better because it is better informed" promise after 20 years. I'm holding my fingers crossed, but I think we need a little more experience with it before victory can be declared.
uh, a misunderstanding, my mistake. what i meant was that graals native-image AOT compiler with the substrate VM achieves 90% of hotspot JIT execution speed pretty much since the project was launched. you may be right that graals JIT is still faster, i don't know about that.
luajit is a great triumph for dynamic compilations and jits; and the fact that it compares favorably to C code (about 70-85% in some of my experiments) is utterly amazing. Especially given that it's essentially the work of one person.
And yet, regardless of how much typing info you give it, it loses out to good AOT compilers, for a similarly implemented algorithm.
GraalVM is unique in that it seems to do better than AOT in some cases. I'd wait a couple more years before declaring JIT victory, though.
The belief is that JITs have information, such as variable values and memory access patterns, that an AOT does not. Well, so far hardly any JIT uses that info, and AOTs do have them from through profile feedback; And also, JITs have to amortize compilation effort and runtime saving, whereas AOT doesn't. So the "JIT is obviously >= AOT" is not foregone conclusion.
Wouldn't it be good if JavaScript also starts supporting python like type annotations (completely optional, but unlike python used to guide the optimizer. Given that some non trivial amount of JS is already generated after type checking (typescript, flow, and all the statically typed languages compiling to JS), all this compile time type info could be made available to the optimizer.
For speed purposes? This post is for example already talking about how fast JS is getting without types.
I'd say it would be a lot easier to simply focus on the WebAssembly format, you can write your code in a number of (typed) languages and compile into wasm which is designed to be fast.
Javascript already comes with a few decades of bagage, and things are already changing really fast (ES6 and newer versions). Not everyone likes types and the people who do can use typescript (or flow). And the people who need insane performance (games, etc) can use WebAssembly.
Yes for speed purposes. And yes some people may not like types, that's why I said completely optional. Anyways the optimizer tries to figure out the types on it's own, so give it a non blank slate to start with. You are correct in noting that js is already fast enough, but the problem is the sort of performance gotchas(order of keys in objects) as mentioned in the post aren't really obvious during development time. Types may help level the unpredictability.
Yeah, I've always wondered this too. It would be cool to be able to extract typings in a separate file, like source maps, and send over the wire along with the JavaScript bundle (for compatibility).
It would definitely help moving sections of the code up the tier ladder, not unlike Web Assembly.
I even predict that future typescript versions would include stricter typings optimized for an browser to solve problems like others have mentioned in the thread.
Or just a TypeScript engine in the browser, like ehm Dartium. I guess someone at Microsoft already did this on IE in a hackathon or something.
I don't think so: TypeScript/JavaScript-Types are too coarse-grained for optimizations: For example number is a double in JS. For generating good machine code you really need to know whether the number is int or double. The same for string: Depending on the JS engine there are actually a lot of different string types.
Even if we consider that good enough or somehow solved, understanding type annotations only helps you reaching peak performance faster. It doesn't really make peak performance faster.
I think the type annotations themselves don't do anything, but the typing strongly pushes developers towards monomorphic (or at least non-megamorphic) calls sites; and having proper-ish data types may also encourages consistent order of initialisation.
I don't really know about the details fo Pypy, but from what I gathered from performance talks about V8 between the hidden classes and inline caching it would strongly benefit from these behavioural changes.
Basically, it doesn't preclude the runtime from having to do its analysis, but it makes the analysis & optimisations "hit" at higher rates than they would otherwise do.
If they really want to redistribute, they could reduce prices. They could return to shareholders, sure, but returning to the market (of their customers) would be more economically valuable.
Think in terms of natural selection. Behaviors (and personalities) that make more successful choices in that environment will tend to survive in the environment, regardless of whether they know what they are doing.
Most of the bumbling fails. Some of the bumbling works. And so the most “effective” politicians (and organizations) will survive, even if their success has little to do with deliberate scheming.
In this narrow case, I believe there may be a claim that those bits are not “Internet” service. They are private network traffic, within their respective networks.
They are still TCP/IP and/or UDP/IP packets running over the "internet protocol". At what point do you draw the fences between "private network" and the rest of the internet?
