Care to share more details of these companies including how the Ruby ones become profitable after a month with 4 guys on payroll? No offense but I've got to check the facts these days.. :).
Also, don't forget Fred Brooks article about the Second System. I betcha smart engineers on tight budget can hack in Java just as quick as those 4 guys and achieve the same result. My apology, Java today is as productive as Ruby on Rails.
The difference between rewriting a Rails site to Java and an already written system un Java to a more modern architecture is huge. One can slowly move from old Java architecture to the newer one but it'll be trickier from Rails to something else... don't forget the accumulated Ops experience managing jvm based app server, the devil in details for those experienced engineers who have been there and done that...
> Care to share more details of these companies including how the Ruby ones become profitable after a month with 4 guys on payroll? No offense but I've got to check the facts these days.. :).
It was small industry, those were basically the only two companies in it, I don't feel I can. (I've already changed some small details e.g. it wasn't actually Ruby). In any case it's just one person's anecdote - but I bet it's happened a lot.
> My apology, Java today is as productive as Ruby on Rails.
Citation needed. In the programming language shootout Ruby still comes out much shorter than Java, and like I said, the only proven risk factor for bugs is number of lines of code.
> One can slowly move from old Java architecture to the newer one but it'll be trickier from Rails to something else...
When I worked at last.fm this was literally my entire job (well, from PHP). You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder.
> don't forget the accumulated Ops experience managing jvm based app server, the devil in details for those experienced engineers who have been there and done that...
You can hire the experience - Java is literally the easiest language to hire for (I mean, hiring is still hard, and there are a lot of crap java programmers around, but there is a bigger pool of experienced programmers out there than any other language). And you're going to need to hire anyway; your ops team will have to grow, your engineering team will have to grow more, and you absolutely cannot rely on the engineers who bought the first version still being around when the time comes to rewrite; the half-life of an engineer in a startup is not large.
Twitter was the article's choice of example, not mine - and from a business point of view Rails worked really well there.
>the only proven risk factor for bugs is number of lines of code.
I'm sure you can cite that, but it feels wrong in this context.
That is, given the same language, or equally verbose languages, it may be true and applicable. But, if comparing a more verbose language to one that is less so, then the former will naturally yield a higher LOC count. But, does this mean it's more bug prone? There aren't any more logical constructs, flow control mechanisms, etc. The language simply requires more lines to express the same intentionality.
In fact, I wonder how a more verbose language would compare with a less verbose one if we normalized for some verbosity factor. So, let's say language A is twice as verbose as language B, such that when the LOC count of A is twice that of B, we consider them equal (call it the same "Effective LOC").
I actually wonder if, in this case, language B would have a higher bug count per effective LOC, owed to its terseness and/or reduced readability.
This behaviour - downvoting without explanation irritates me too when there is very lucid and concisely structured argument you can oppose.
As such, I do not find your argument for verbosity normalizing convincing :)
I fail to see how normalizing some risk metric against a verbosity of language will provide any valid data in the general context. For numerical algorithms, where a multiplication is just as verbose in any language then there is no difference.
But when manipulating collections, for example in C I need to write
int i;
for(i = 0; i < someSize; i++){...}
This has several places where the verbosity of language can make it easier to write bugs that slow down the progress a bit. If we add these tiny slowdowns throughout the lifecycle of the entire application the cost is not neglible if compared to a situation where these bugs would not appear at all. Some other language:
for i in collection do:...
Which does really have no place to shoot oneself in the foot unless the iteration logic itself does something funny.
Well, I can appreciate the reply versus a downvote, but "I do not find your argument [against] verbosity normalizing convincing." :)
>I fail to see how normalizing some risk metric against a verbosity of language will provide any valid data in the general context.
OK, then let's look at the corollary: that is, not normalizing against verbosity would mean, roughly, that you're claiming that a language that is twice as verbose would yield twice the bugs over the same app. I don't think that's the case nor what you intend to say.
I think you're also confusing complexity with verbosity. An extremely terse language can be very complex, and may even suffer from readability issues.
Overall, the notion of LOC alone being a risk-metric absent any additional context seems an oversimplification.
"Overall, the notion of LOC alone being a risk-metric absent any additional context seems an oversimplification."
Yeah, I agree completely. Didn't think too long before answering there, I'm afraid. I confused the fact that in my experience increase of LOC correlates with increased risk of bugs with there actually being a causality between the said observables.
For me personally languages with higher LOC are harder to read, but this is just about the languages I like and not a generalization that can be applied to any language in general (i.e. Ocaml is nicer than C in this sense but APL or J would get me just terribly confused).
Basically, of the languages I consider readable, and when dealing with container operations, it's easier for me to parse especially those that either have basic containers as fundamental data types in the language (e.g. Python, F#) or have a uniform syntax over typical algorithms and containers (Clojure) rather than verbose languages like C or Java.
In these instances, it's not about the overall LOC count of the languages but the fact that they make common operations more legible and easier to reason with than C. I.e. they have less non-value adding burden typically. Why they are nicer, is not because of their LOC, though, but their LOC is a side-effect of their features that make them nice, I think.
What you argue is not that LOC in itself causes bugs, but that certain programming syntaxes are both complex (thus induces bugs) and takes up more LOC.
>You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder
I have to disagree here. With just a little forethought, a good bit of the code can be reusable. For instance, your data access layer.
And, the more forethought you put into it, the larger the potential reusability. This doesn't have to be premature optimization. It's just a balance between getting it done as quickly/efficiently as possible while still making some sensible, fairly low-cost coding decisions for scalability. For instance, everything I write now is load-balancer ready. This was achieved largely through a simple library that understands its environment via a simple config directive, and proxies relevant calls (e.g. when pulling the client IP or checking whether the request is secure, it determines when to pull from the request headers stamped on by the load balancer versus invoking request.getRemoteAddr() or request.isSecure()). Of course, there are other coding decisions you make, but it's really not as much extra work as how you implement.
In any case, unless you're purposely writing completely nasty throwaway code, there is always some reusability, which typically makes starting from scratch in a new language a much weightier prospect than making some scalability enhancements to the existing base.
> In any case, unless you're purposely writing completely nasty throwaway code, there is always some reusability, which typically makes starting from scratch in a new language a much weightier prospect than making some scalability enhancements to the existing base.
It's not really from scratch. If you've figured out the correct structure and interfaces to solve your problem you can reuse all that when porting to a different language, and that's the hard part of development.
>If you've figured out the correct structure and interfaces to solve your problem you can reuse all that when porting to a different language
Well, I think you're underestimating the sheer brute force work needed to port an app with a large-ish LOC base.
But, beyond that, if you've gone through the care of structuring it correctly such that the structure and interfaces are preservable once scalability needs arise, then all the more reason not to port it. Presumably, at that point you'd have, not only the structure, but working code.
So, if it can be so readily adapted to a new language and you're preserving so much code, then why bother porting it? Kind of makes may original argument. Just apply the scalability tweaks on the existing code base and you're done.
> Well, I think you're underestimating the sheer brute force work needed to port an app with a large-ish LOC base.
I have done this professionally. You don't port everything, you just port services as they become bottlenecks. By the time you need to scale like this, you already need to have multiple engineering teams and you need well-defined interfaces between components, so having parts of your system in different languages doesn't impose an additional cost.
> So, if it can be so readily adapted to a new language and you're preserving so much code, then why bother porting it? Kind of makes may original argument. Just apply the scalability tweaks on the existing code base and you're done.
Sure, which is probably why Digg was happy to carry on using PHP. For the vast majority of systems you never get to the scale where using Java is worthwhile (at least in performance terms).
When do you want to port? When your server costs get high enough that a 15x performance factor is worth the constant cost in dev time that using a lower-level language would cost[1]. It might take more effort than writing in Java from day 1 - but it might not, if you go through several iterations of prototyping/refactoring in PHP and then only port once you've found the correct design. And more importantly, you get to pay the Java price much further down the line, when you're bigger and more profitable. As PG says, an early-stage startup should do things that don't scale.
[1] I'm not convinced you can't have your cake and eat it, with e.g. Scala or Haskell. But assuming for the sake of the argument that the only languages were Java and PHP and PHP really was more productive than Java but less performant.
>I have done this professionally. You don't port everything, you just port services as they become bottlenecks.
Yep, been there too. Except, in the case I'm thinking of, it wasn't for performance. We went with a multi-language approach for different reasons, and we ended up with a hybrid that no-one liked. Then, we finally decided to port everything to one language.
All of this to say that a multi-language approach can have its own drawbacks, which is not to say that it's never appropriate. There should just be a really good reason for it. Generally, I'm not sure that starting with a plan to half-port a non-performant app is a good reason. But, I'd never say never.
I'd also add (again) that if you've really taken the time to structure the app so well that you have clean interfaces and modularized, easily-portable services, then it begs the question: did you really save so much time by then implementing that design in PHP vs Java? As you said, the design is the hard part, and I'm just not seeing the notion of a quick non-scaling app, somewhat hastily implemented in PHP, being compatible with the idea of a tight, service-oriented, modular design. I think you kind of have to choose one.
I don't suppose any approach is necessarily the absolute right approach per se. But, given that the article is about picking one approach/language (Java) and sticking with it, I'd have to say that I'm still convinced that it may indeed be optimal, even after reviewing your points.
> I'm just not seeing the notion of a quick non-scaling app, somewhat hastily implemented in PHP, being compatible with the idea of a tight, service-oriented, modular design. I think you kind of have to choose one.
Oh sure, the initial code was pretty terrible. But by the time we were moving parts to Java it had all gone through a lesser or greater amount of refactoring; the best design is iterative and emergent, and we had the http://c2.com/cgi/wiki?WhatIsAnAdvancer experience several times over. In the early stages you really can iterate faster in not-java; as the correct design emerges and the codebase gets larger the interfaces tend to firm up, and at that point it becomes more reasonable to port pieces for performance.
> like I said, the only proven risk factor for bugs is number of lines of code.
There's never been a proof that writing code in Ruby WILL give you LESS bugs than in Java just because you'll end up with LESS LOC. I think this is a generic statement that needs to be clarified.
I agree and I will not argue that LESS lines of code "might" give you LESS bugs given that you're writing it using the SAME language (version 1 in Java has more lines than version 2 in Java, maybe because in version 2, the developer gained more experience and can write idiomatic Java and use libraries instead of rolling his/her own). But Ruby yielded less bugs because of less line of code is a marketing speak to sell the religion of Ruby.
Sometimes I wonder why developers aren't forced to just use LISP since everybody in the whole world knows that LISP is super expressive that it MUST resulted in far less code right? Right?
> When I worked at last.fm this was literally my entire job (well, from PHP). You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder.
I work for a company that has been using Java since probably 2002/2003 (I joined in 2012). Server-side in Java with Client-side in Swing, slowly evolving the Server-side POJO (probably no Application Server), to EJB 2.x (shudder) to EJB 3.x and goes to the cloud we go (AWS). Recently we refactor a certain part of our code and added Microservice written in JAX-RS because we have to integrate with another product suite internally. That's what I called re-use and evolution of code-base: 10 years of Java where some of the code from the past were/are reused and still in production today. (.... but boy those old code aren't pretty, hehehehe).
Oh, I almost forgot the part where we used shell script to build, moved to Ant, moved to Ant + Ivy, and slowly moving to Maven (if we need to write a new microservice, maven by default, the rest have to move slowly).
Oh, here's the best part: last year we moved away from homegrown migration tool to use Flyway, 4-6 lines of Java code to initialize and execute the migration workflow dropped in our old Java code.
So yes, if we had to re-write and re-write and re-write to different languages every time, it is definitely will be harder.
> You can hire the experience - Java is literally the easiest language to hire for (I mean, hiring is still hard, and there are a lot of crap java programmers around, but there is a bigger pool of experienced programmers out there than any other language).
Ops, not devs.
> and you absolutely cannot rely on the engineers who bought the first version still being around when the time comes to rewrite; the half-life of an engineer in a startup is not large.
Thank God it's Java, not PHP, not Perl, not Ruby (I give Python a pass because I'm being nice). IDE helps a lot. This reminds me of Zed Shaw blog about charging a lot of money fixing Rails mess as a consulting service.
> Twitter was the article's choice of example, not mine - and from a business point of view Rails worked really well there
I'm not suggesting that they made a bad move or whatnot I'm just merely debunking the common stories of how small team beat big team (that may have big expectation/big scale yet not focused on solving customer needs) because of the technology/languages. Talent is something else. Tech/Languages/Scaling? sounds like ego problem there.
> Sometimes I wonder why developers aren't forced to just use LISP since everybody in the whole world knows that LISP is super expressive that it MUST resulted in far less code right? Right?
You've fallen for Lisp propaganda :P. If you look at the shootout, Python and Ruby are comfortably ahead of Lisp in the expressiveness stakes.
Really not. This was one of the leanest places I've ever worked; everything we did there was absolutely necessary for the scale we were operating at. I've seen second-system effect in action; last.fm wasn't it. Writing Java will not save you from having to scale horizontally; maybe it's 2x, 5x, 10x faster than a dynamic language, but if you're growing then these kinds of constant factors only buy you a small amount of development time.
> Recently we refactor a certain part of our code and added Microservice written in JAX-RS because we have to integrate with another product suite internally. That's what I called re-use and evolution of code-base: 10 years of Java where some of the code from the past were/are reused and still in production today. (.... but boy those old code aren't pretty, hehehehe).
If you're using microservices multiple languages really shouldn't be a problem. It's not like we rewrote everything; services that were fine in PHP stayed in PHP. We moved things to Java as and when we needed to. If everything's a service (Thrift in our case) then it really doesn't matter what language is on the other end.
And it's not exactly "from scratch". Physically typing out the code is never the bottleneck; most of the hard part of programming is design, getting the correct decomposition of the problem. Porting well-written code from one language to another language that supports the same paradigms is really not that hard, and certainly doesn't mean throwing away all the lessons of the first codebase.
Also, don't forget Fred Brooks article about the Second System. I betcha smart engineers on tight budget can hack in Java just as quick as those 4 guys and achieve the same result. My apology, Java today is as productive as Ruby on Rails.
The difference between rewriting a Rails site to Java and an already written system un Java to a more modern architecture is huge. One can slowly move from old Java architecture to the newer one but it'll be trickier from Rails to something else... don't forget the accumulated Ops experience managing jvm based app server, the devil in details for those experienced engineers who have been there and done that...