I think it's a matter of ontology. The category "ML" is pretty fragmented, but the categories "Haskell," "F#," and "OCaml" are all pretty non-fragmented. Ditto for "Lisp" vs Clojure, Scheme, Common Lisp[1].
I don't think Ruby should be compared to ML. I think Ruby should be comared to OCaml. ML, if it gets compared against anything, should be compared against the category that includes Python, Ruby, Perl, and JavaScript. This category is about as fragmented as Lisp.
The problem with this new category is it doesn't have a simple, clear definition. Lisp is pretty much defined by S-expressions, and MLs by their type systems. This new family might be defined by their common approach to object systems (i.e. objects as dicts), although that's really more representative than fundamental.
[1] I'm under the vague impression that Common Lisp and SML are somewhat more fragmented than other members. I wonder if this correlates to their being the "Common"/"Standard" dialects.
> Ditto for "Lisp" vs Clojure, Scheme, Common Lisp[1].
Clojure is not fragmented, but both Scheme and Common Lisp are extremely fragmented. Scheme is probably one of the most fragmented language there is, there are literally dozens of implementations of it with various levels of compatibility[0], and it's actually gotten worse lately as many (if not most) implementations refused to migrate to R6RS and remained on R5RS instead. There are even implementations (e.g. DrRacket, formerly DrScheme) which forked/opted out the whole language.
From the main Scheme article on Wikipedia: "Scheme follows a minimalist design philosophy specifying a small standard core with powerful tools for language extension. Its compactness and elegance have made it popular with educators, language designers, programmers, implementors, and hobbyists."
Scheme is widely (mostly?) used as a teaching and research language in Comp. Sci. schools. This sort of environment encourages experimentation with the language itself, whereas Python and Ruby are more oriented towards practitioners who are more interested in using the language as-is, not experimenting with it.
I don't know that I'd agree that Common Lisp is "extremely" fragmented. It's true that there are probably more implementations than we really need, but they do serve different niches. Anyway, while "extremely fragmented" might have been a fair characterization a few years ago, there has been enough work done lately on portability of libraries -- much of it driven by Zach Bean's Quicklisp -- that switching implementations is often straightforward now.
CL is defined by a standard which most of the implementations work pretty hard to adhere to. Of course, most of them have their own extensions, but a lot of these (particularly in the important areas of multithreading and foreign function calling) are now abstracted by portable libraries. This is a very different situation from the Scheme world.
Racket didn't opt out of R6RS. Racket has multiple language modes, one of which is actually R6RS Scheme. It also has a "legacy mode" for R5RS, along with modes for a handful of other languages such as Datalog, ALGOL 60, and some teaching languages.
I'm familiar with the name change, but they didn't opt out of Scheme at all. Racket still supports both R5RS and R6RS. And before someone starts going on about the "default language" not being Scheme anymore, Racket doesn't have a default language. You have to specify one. Many of the things that have been added to Racket beyond the Scheme standard also have analogs in other Scheme systems.
"scripting languages" was the commonest blanket term. then it fell out of favour and people usually refer to "dynamic languages" now, which is a lot broader a term and doesn't really single out just that family.
I think the problem with this question is that "fragmentation " is fairly ill-defined.
For example, you can say that ML fragmented into SML and OCAML, but then what's Haskell? Is it a fragment of SML? Well, not really. But then, SML and OCAML are significantly different languages, which have taken their inspiration from sources outside of their "parent language" as well.
On top of this, look at, for example. Boo. Boo is a lot like Python, but is it a fragment of Python? Also, C, C++, C#, Java, now even Go.
I think the point of this is what is the point at which a new language becomes no longer a "fragment" of its parent, and becomes a new language in its own right?
<blunt>because python and ruby as just language(s| concepts) aren't worth much. The only people that saw enough worth in implementing them are the ones that designed them. Their worth now comes from libraries/ecosystem/community and initially came from pragmatism/simplicity which also came from them probably being designed/implemented/used at the same time by one person.</blunt>
JUST MY VIEW:
Python probably didn't get popular when Guido released it. I was using it 10y ago and it already had a lot of libraries and "batteries included" then, but it was still far from mainstream.
I think it got mainstream after the RubyOnRails Wave that made a lot of (mainstream) programmers (that only use what other use and/or is popular) accept dynamic languages. And when BackSlash came (Ruby is Slow, RoR is full of Magic) python w/ django was the dynamic lang that clinged on shore. Not accidentaly since it already had pretty clean core and a lot of libraries (it always had more libs than ruby). Now everybody uses it.
If you look at the similar languages w/ a lot of libraries and bindings php/Perl/Ruby/Python it's not weird that it's popular and I would say that it is the simplest/cleanest.
Yes, Python was strong before Ruby (which I indicated hopefully), but it wasn't as mainstream as it is now when it's everywhere (at least in circles around me).
I back then used it for a lot of things and most weren't related to web. But I think it made the biggest jump after Ruby backslash. From quick peek at TIOBE:
""" "Programming Language of the Year" award winners is shown below. The award is given to the programming language that has the highest rise in ratings in a year. """
Mailman, the original BitTorrent client, Gentoo's Portage/emerge tools, yum, Eve Online, and a large swathe of 90s-era Red Hat Linux system configuration tools have all been written in Python, all predating Django and your "Ruby backlash" timescale by at least 2-3 years. And not one of them is web-related.
Most of the code that gets written in Python has nothing to do with web development. Web development is just high-profile.
Perhaps you are putting too much emphasis on the Python world's support for programming web apps. Python is also a good programming language for many of the jobs we used to use Perl for, but with much cleaner syntax in a lot of cases and with more batteries included. That is how many of us first discovered it.
In the case of Python, there are such dictators. With Ruby, it was less the dictator and more the dictates of a poorly defined language specification. "The spec is this running C code" is thought by many to be a smell. (Also said for VP8.) In the case of Ruby, multiple groups of several people each worked for long stretches of time (half a year or more) to come up with parsers and non MRI specifications for Ruby syntax.
Conversely, one can implement Lisp or Smalltalk syntax quite quickly, which means the barrier to fragmentation is pretty small. Python is still fairly small. I think that there would be more fragmentation without its benevolent dictator.
This really just seems like a matter of definition. Rather than viewing LISP dialects as fragmented forms of one original language, we can view Common Lisp, Racket, and Clojure---the three main forms of LISP seen in "the real world"---as three separate languages that, like Python and Ruby, haven't fragmented. We can view the fact that all three use the LISP model of computation simply as a design choice and not a sign of "offshootness", in the same way that we don't say that Python and Ruby are the results of ALGOL's fragmentation.
> we can view Common Lisp, Racket, and Clojure---the three main forms of LISP seen in "the real world"---as three separate languages that, like Python and Ruby, haven't fragmented
As far as Common lisp and Scheme go, there are things outside the rnrs and common lisp standards that cause the fragmentation.
I don't use common lisp so I can be wrong but going by the specs, there is no mention of threading, networking et al.
I'm not a CL expert, but the existence of multiple implementations of CL doesn't appear to have caused much fragmentation. The language did evolve beyond the standard (e.g. threading and the meta-object protocol), but if I'm not mistaken all popular implementations of CL (SBCL, Clozure CL, Allegro, LispWorks, CLISP, CMUCL) implement the full meta-object protocol, and even if their thread implementations use different libraries, isn't that pretty common among programming languages that aren't considered fragmented? Different implementations of C are certainly like that. Unless the OP really meant to ask why some languages have one main implementation and others have multiple, but that seems like a very different question (whose answer in the case of CL is obviously not that it broke off into multiple languages but rather that it was designed as a standard without any official implementation to begin with).
One significant difference between (Ruby, Python) and (ML, Lisp) is the nature of the world they were born into.
Widespread internet access, along with the tremendous utility that the internet holds for developers, strongly encourages the concept of a canonical implementation of a language, in a way that simply didn't seem nearly so important 20+ years ago.
(It's also interesting that the internet encourages this notion of canonical implementation even while simultaneously encouraging the creation of forks.)
I can speak to why there are alot of implementations of Lisp and Scheme.
Scheme seems fragmented because a lot of research work use it as the primary tool. For example, you have languages like Larceny which were primarily designed to study Garbage collections or even probabilistic language models using Church Scheme.
The ease in which Lisp and Scheme allows you to create new languages and perform exploratory work has spawned a lot of newer implementations.
And People also come up with newer implementations for a specific goal in mind. For example, Bigloo Scheme was created in order to enable Scheme programming style where C(++) is usually required.
I would phrase that differently: for many of their early 'users', Lisp was the object of study, not their tool. If your main question is not "I want this functionality, ASAP", but "how would I improve this tool?", it is normal to build a new, improved tool, rather than some app or library.
Ruby, perl, python, C, etc are tools for most people. People who do use them as study object will improve them by inventing a new language, rather than by tweaking them and still calling them by their original name. The main reason for that, I think, is that it is about as easy to implement a new language as it is to tweak those languages.
Ruby and Python (and Perl, etc.) are languages for pragmatists. Their user communities are overwhelmingly dominated by folks who've got a practical problem to solve, and primarily see the language as a tool to help solve that problem. They also tend to be rather "CISC"-y, which makes trying to customize them less attractive because complicated things are harder (and often less fun) to tinker with.
Lisp and ML (and Forth, etc.) are languages for language geeks. They might see a lot of practical use, but they're also popular as a solid working base from which to experiment with new language ideas. And they benefit from being fairly small and orthogonal. Most fans of languages in this camp have written their own implementation of their language of choice - partially out of being a language geek, and partially because their specs are clean enough that doing so isn't a very intimidating prospect.
Ruby has multiple implementations, but none of them can deviate too far because of Rails. It's like the old test of IBM PC compatible computers, can it run Flight Simulator / Lotus 1-2-3.
You could say that both Python and Ruby are instances of Lisp's fragmentation, rather than unfragmented alternatives. (I am not a Lisp programmer, BTW.)
Python and Ruby are both very fundamentally Not Lisp: Lisp's intentional s-expression syntax is very much a part of what makes Lisp powerful (it lets you write macros very easily, since code and data have the same representation).
Python and Ruby both don't have that. They are alternatives to Lisp, and in the same family of dynamically typed programming languages, but they are not "instances of Lisp's fragmentation" in any way: nobody took a Lisp-like language and from that created Python.
Python and Ruby have first class functions, garbage collection, dynamic typing, and many other advanced dynamic features, so there is definitely a similarity between them and Lisp. However, the fact that they don't have code as a first class object is definitely a major limitation. In Lisp essentially everything is first class and manipulable at run time.
Sure, they are similar. It would be very strange, however, to argue that they are fragmentations of Lisp in the sense that Ocaml and SML are fragmentation of ML. They may share ideas since they are in the same semantic family (dynamically typed) but that's about the extent of it.
Lisp's s-expression syntax was very much not intentional: IIRC McCarthy specifically wrote that he left implementation of the syntax as an excercise for the reader. As it turned out, very few readers chose to devise a syntax and instead worked on s-expressions directly.
I am dubious of this assertion. The original 1960 paper "Recursive Functions of Symbolic Expressions and Their Computation by Machine" introduces S-expressions and M-expressions as explicit entities (not as some abstract syntax, but as a literal syntax that people were able to write down). Now, M-expressions never really got used because everyone just used S-expressions, since they were simpler.
Do you have some citation about how he left implementation of syntax as an exercise? I'd be interested to see that.
I think he's kind of close, at least in that it was not meant to be written as S-expressions. AFAIK, the M-expressions were meant to be the main syntax. The very early Lisp docs I've seen were all written in M-expressions, with S-expressions taking the lead something like a decade later. (This was all decades before my time, so it's possible this is sampling bias, but I don't think so.)
Python and Ruby are very different from Lisp. Other than the fact that they share certain ideas that originated in the Lisp community ( GC for example ) the languages couldn't be farther apart. Programming in say, Ruby, is a vastly different experience than Lisp.
Interesting take. I'm not a Lisp programmer either, just know enough about it to be dangerous, but it seems to me all programming languages, including Lisp implementations like CL and Scheme, are mere instantiations of Lisp, with (byte) compilers merely parsing down into a syntax tree that a Lisp-like engine can then execute.
...it seems to me all programming languages, including Lisp implementations like CL and Scheme, are mere instantiations of Lisp...
There's an element of truth in that, but only in the same sense that all Turing complete programming languages are equivalent or that many programming languages are ultimately instantiations of a register machine, a memory store, and a program counter. The interesting thing in most cases is what you build on top of that foundation.
With Lisps, the value is in the uniformity. That makes it very easy to extend the basic functionality you get out of the box in a way that fits in, using macros and so on.
With various other languages, the value is in the pre-built tools that rest on top of some underlying abstract syntax tree. That covers a vast range of functionality these days: more expressive type systems, a wider range of built-in data structures, actor-based concurrency, low-level memory and port access, and numerous other practically useful language features. Hopefully any given language offers a combination of features that work well together and have natural supporting syntax and semantics. That in turn gives each language its own flavour and means each language works better for some types of software than others, even if ultimately it's usually just syntactic sugar for a relatively simple execution model and an AST.
Two things: both mtts and Silhouette are expanding on what I meant (rather than what I wrote):
McCarthy's Lisp:{Python,Ruby}::zygote:adult human
mtts as quoted by Silhouette gets it right IMO. Conversely, S's distinction between extending a language and using the tools (the distinction, to oversimplify, that makes you a Lisp programmer) that sit on top of its core seems to me a confusion of means with ends. Any useful program is a fall from the perfection of JM's insight. This perspective makes me a not-Lisp-programmer. Thousands of developers smarter and more experienced than I have taken each side of this dispute without resolving it; I certainly don't hope to.
I think perhaps you're suggesting a false dichotomy there. Almost any useful general-purpose programming language is going to have both some degree of extensibility and some degree of support from built-in language features. The different flavours are a question of priorities, because the more specialised tools you provide via dedicated language features, the harder it is to keep the core of your language generic. The really successful languages tend to have a set of specialised features powerful enough to give a significant advantage over languages without them, but well enough matched and with clean enough semantics that the overhead for extensibility in other directions isn't prohibitively increased.
> ... it seems to me all programming languages, including Lisp implementations like CL and Scheme, are mere instantiations of Lisp, with (byte) compilers merely parsing down into a syntax tree that a Lisp-like engine can then execute.
Lisps allow you to extend both the parser (reader macros) and do transformations on the syntax tree (macros) before it gets compiled/interpreted. In most of the other languages parsing and code generation are very tightly coupled with no easily accessible step in between.
What is important in the read-eval-print loop is the dashes :)
One big difference to me is that many of the ML implementations were developed (and are still developed) to research a particular technique or problem by competing academic groups. From what I can tell Python/Ruby are not really used for research in this way and most of the forks are to port to different platform/interpreter or to make a faster implementation (PyPy).
I don't think Ruby should be compared to ML. I think Ruby should be comared to OCaml. ML, if it gets compared against anything, should be compared against the category that includes Python, Ruby, Perl, and JavaScript. This category is about as fragmented as Lisp.
The problem with this new category is it doesn't have a simple, clear definition. Lisp is pretty much defined by S-expressions, and MLs by their type systems. This new family might be defined by their common approach to object systems (i.e. objects as dicts), although that's really more representative than fundamental.
[1] I'm under the vague impression that Common Lisp and SML are somewhat more fragmented than other members. I wonder if this correlates to their being the "Common"/"Standard" dialects.