Accelerating scikit-learn is a smart move. At the algorithmic level for every ML use case there is probably x 10 non-ML data science projects. Also, it is good to have a true community framework that does not depend on the success of the metaverse for funding ;-)
The lock-in is an important consideration, but if the scikit-learn API is fully respected it would seem less relevant. It also suggests a pattern for how other hardware vendors could accelerate scikit-learn as a genuine contribution?
Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly.
Julia is one of those "nice in theory" options which has failed to live up to the hype and at this point seems unlikely to unseat python for most use-cases; it just doesn't have a good enough UX when used as a general purpose language.
im not sure what you mean by UX in this context, but julias ecosystem for scientific computing (in a broad sense) has been growing tremendously. this is the area from which it wants to unseat python. general purpose programming is secondary. whether it can i don't know. but i definitely dont think its a settled question. python is my daily driver for machine learning work, but i definitely think julia can overtake its place eventually
The editor autocompletion and standard library documentation could use a lot of work. The introductory tutorials are overly focused on type theory and details and do not give a good overview of which generic data structures to use in production code. Julia's JIT is very different from other conventional mainstream languages and the process of selecting standard library generic data structures for optimal performance is very poorly documented.
There is no Effective Julia style of guide. You either have to wade through infantile tutorials for those with minimal programming experience or several reference books worth of nitpicking on syntax. The actual methods themselves are not well documented and lack examples and usage guidelines.
The language and ecosystem do not feel like a project backed by commercial funding, it feels like one of those functional languages out of academia research where the structure and design of the language are more important than actual developer experience. There are many new projects but most are not actively maintained and updated. The language itself feels massive, with syntactic sugar and weird types everywhere. Trying to understand the implementations of other people's Julia code is frustrating, similar to reading a library written in pure C++ templates. Compared to Go/Rust/Dart, Julia feels overly convoluted. Julia literature is structured in a way that seems to heavily encourage you to take regular classes and lectures to learn and pick up the language. It is hard to feel productive from the get-go.
Someone else commented with more detail, but personally I can't get past the package management and the dependency on using the REPL. Rust gets tooling and packages right.
exactly right. REPL is a key feature for computational scientists. in physics, for example, the value proposition of python was that it provided an open source alternative to matlab (also a REPL based environment) without sacrificing functionality. i believe that the really revolutionary thing with python was that it provided an extremely fertile ground for open source development of numerical methods that far exceeded what was offered on matlab in syntax that resembled pseudo code (much like matlab). julia's value proposition is all of that, plus a much more performant base language with arguably even better syntax
Is that REPL usage still relevant in a world of Jupyter notebooks?
Getting started with Julia always just feels clunky to me - perhaps the other commenter was closer to the mark in blaming the documentation rather than the REPL itself. Either way, despite being a former scientist who has moved into IT (sadly), I get the distinct impression that the language is just not aimed at me. As such, I'm always surprised to see people trying to push it in settings outside its current realm of adoption; feels very much like the language maintainers have no real interest in that.
REPL is pretty similar to jupyter. Julia works well with jupyter, but if you like notebooks, you should definitely look at Pluto. It's a reactive notebook so it automatically tracks cell dependencies and makes it so your notebook is never in an inconsistent state.
I actually kinda do like Julia's syntax, but the OP's comments about poor introductory documentation for experienced programmers definitely ring very true.
I don't dispute that, but for people not noodling around with data who just want to make something immediately useful (ie general purpose programming use-cases), the Julia approach to dependency packaging is sub-par.
The lock-in is an important consideration, but if the scikit-learn API is fully respected it would seem less relevant. It also suggests a pattern for how other hardware vendors could accelerate scikit-learn as a genuine contribution?