The issues about the lack of tooling and stability are well heeded. Unfortunately, we at the Julia Lab at MIT, like our colleagues that are the JuMP group at the MIT ORC, are limited to working on things that can be justified as research activities. Developing new algorithms and applying them to scientific computing activities is research; writing a debugger and testing framework is not. For non-research development we have had to rely on the goodwill from hundreds of open source community developers, which have helped improve Julia and its package offerings immeasurably.
Happily, the present situation is more rosy than it has ever been. Funding agencies are beginning to realize that academic software development is an activity worth funding in its own right, not necessarily tied to research. Julia Computing has also been fortunate to receive funding from organizations such as the Moore Foundation to work on precisely the tools that users most need, but are difficult to justify development with research funds.
Thanks you too for your contributions. From an enterprise perspective, we used a pre-release language with the expectations of a stable production language, so we knew to expect these instabilities.
I'm glad julia was useful for you guys. Of course huge kudos goes to Miles, Iain and Joey (and the rest of the JuliaOpt folks) for building JuMP. I'm sorry you guys didn't end up being able to use it in production, but I can guarantee you that we (both as the open source community and as Julia Computing) are feverishly working on improving the language and tooling to get ready for a stable release. Thanks for being an early adopter!
Thanks for your contributions, Keno! We saw major improvements in the language and its libraries as we were using it. Unfortunately, the effort required to update our codebase for new Julia versions while developing new code proved to be too much. I'm looking forward to a 1.0 release!
I use Julia extensively for scientific research. It's such a great language though that I wish I could use it for more than that. I find that writing Python code irks me after programming in Julia for a day.
I almost feel like there should be two versions of Julia: a commercial version that focuses on stability and is usable in production systems, and a scientific version that undergoes constant revision/feature overhaul to produce the best programming language for numerical computing.
I've been messing about with Julia for a couple of years, I did a few tech talks about it at work and wrote a few side projects in it.
At work most people agreed with the technical insights - it's just a super language, but everyone thought that it was impossible to use it in any real project and very quickly discarded it even as a hobby tool. The clue is / was in the versioning - 0.3; 0.4 now 0.5. Things are broken and major things will change.
I think that this is really good, and important. Getting Julia right may really matter in the future. Code that costs less to write, runs faster and is easier to maintain is an underpinning capability for our economy and for science. Julia could be a huge enabler.
Having said that I find myself having a meltdown over 0.5 (specifically I had a tantrum yesterday when everything broke). I wanted to do some stuff on a private laptop so set up an env under windows. I jumped versions from 0.3.9 to 0.5.3478 because - why not? I'd also been using a 0.4 dev version sometimes, it worked ok but there was one issue that made me normally run 0.3.9 which was fine. However, Atom couldn't cope because HttpServer is @unix_only (whatever the that means, even the complier didn't know, it just said "documentation error")... I had about 100 compile errors in my code due to "' ' not allowed before '(' in foo (bar, bee)" but ok I just fixed those, one of the things that attracted me to Julia was whitespace liberalism, a minor irritation with Python, I find it easier to read code with a bit more whitespace. The trip wire was that I got the same thing from dependencies - like DataStructures.jl
This is intentional by the devs (I put an issue on GitHub and it got closed with that note). I do like "lets get it right", but even as someone just messing about with prime numbers in the evenings I can feel the pain of anyone who is trying to do more significant things. On the other hand a fork like the one you describe is going to be bad - Python 2/3 bad, and Julia is in no way as established as Python was when it had it's fork. The consequences are that the stable libraries are going to have to focus on the stable build and the problem is that at that point the dev version is irrelevant.
For Julia that is a big problem. There is a lot to do in terms of developing the core infrastructure in the language, see the idea of a registry in 0.6 or shrinking base and having a standard library for example. I do think that there is a worry that in covering off all cases and requirements the lovely simplicity of Julia may get diluted, the registry is especially a worry here because dealing with config files and issues is one of my bugbears and I was hoping Julia would stop this nonsense by "just use Git".
Still - I picked up Java at 0.7 and it really became useful at 1.4 Julia is on a much better trajectory already!
Do you know, I simply don't remember - maybe it was 1.0, but I have a feeling that I downloaded a beta and was overjoyed at being able to scrap all my hoaky c++ sockets / CORBA code and reimplement in something amazingly easy called RMI !
Not sure why I thought 0.7, perhaps I was wishful thinking wrt. how much change happened in early Java, but I think that the conventions of version numbering hadn't settled in the same way as they have now.
I'm really impressed you used Julia in production. Despite its immaturity and all the reasons you mentioned, building your product on it is a testament to the language, its potential and the core group working on it. I wonder what the performance differences in your product are going from Julia to python?
I've had my eye on it for the last two years and really look forward to its 1.0 release. There is no doubt there is an audience for it!
I'm currently using julia to write automated continuous testing for a computer chip. My only beef is that it isn't zero indexed.
I probably could have done it with python, but IMO the language just is cleaner and easier to read. The macro system helps, a lot. The typesystem also gives me confidence in the correctness of some of the stranger code I'm emitting.
It's a really great prototyping language for mathematical concepts.
Yep, waiting for 1.0. I really like Julia. Zero-indexing would be preferable, or at least a switch like APL to choose what your default indexing is. It's easy to switch if you use somebody's else's code that is 1-based.
Our Julia library did shift generation and shift assignment in one step. We split these tasks into two separate microservices as we transitioned away from Julia. So, it is tough to compare benchmarks. We are using some compiled libraries in Python that make these calculations parallel.
I was playing with Legos recently and thought of a new way to approach the shift decomposition problem, and I'm amid a prototype in Go that is promising.
We're not using column generation in the current algorithms (but Leanform is just launching with column generation on top of CBC - https://angel.co/leanframe ).
We're not using Dantzig-Wolfe decomposition, but our original scheduling algorithm used a dynamic programming approach that was quite similar.
We're currently amid rewriting the Chomp decomposition algorithm to be a more specialized branch and bound variant.
Yes do do have a column generation algorithm that leverage my expertise I gained working at ad-opt (airline pilot rostering).
We also migrated from julia to python. Another vote for the MIT teams that worked on the project! It really helped us at the beginning!
We are also playing with the google glop solver that gives pretty good results so far.
The main drawback of python for column generation (and as far as I understand, the same flaw exists in juliaopt/JuMP) is twofold. 1) most libraries are "math formulation oriented" which makes implementation hard to maintain/debug 2) biggest issue: the wrappers around the underlying solver (cbc, glpk, etc) keeps all references (to variables and coefficients) and do not offer a clean add/remove interface (only add/set bounds). Even playing with coefficients isn't always straigthforward.
So we had to build our own wrapper on top of the LP wrappers out there (Pulp, pyomo, cylp etc). When solving large scale problems with column generation, you generate hundreds of thousands of different variables even if only a fraction of them are "active" at a time leading to memory and runtime leaks in the usual wrappers. We had cases where the LP library was spending about 85% of the time just writing and reading files for CBC. The or-tools library wraps the C library itself via SWIG and is our best horse as of writing this.
For sure we will think of rolling our own binder too. Last year I was hoping to do it is scheme :) We might get "stuck" with python for the time being though.
Saw that dependency management was going to be a pain from the get go. Suggested a fix, was rejected. I decided not to use Julia b/c of this deficiency even though overall I like everything else about the language.
Yes, thanks. Seems I didn't finish my post before I hit submit - someone came into the room to collect me and I had to log out quickly. By the time I noticed the edit window was closed.
Testing support is basic and, when we started, there were no clear best practices - so we published how we organized our repo.
The `@test foo` command basically raises an exception if the result is false. You have to manually register all of the tests and call them individually. setUp and tearDown has to be done manually. The testing library keeps no data about how many tests were run - it just throws an exception if an `@test` encounters a false.
This is partially fixed in Julia 0.5 (or available now as BaseTestNext.jl in Julia 0.4). Its basically a backwards-compatible rewrite of the former Base.Test that adds the notion of test sets - enabling counting tests by default, but also not immediately failing out on the first bad test and even logging.
At the core, most programming languages are quite similar, but the developer tools and communities shape the culture and ultimately the roadmap of the language.
Happily, the present situation is more rosy than it has ever been. Funding agencies are beginning to realize that academic software development is an activity worth funding in its own right, not necessarily tied to research. Julia Computing has also been fortunate to receive funding from organizations such as the Moore Foundation to work on precisely the tools that users most need, but are difficult to justify development with research funds.