> In other words they would have already proven themselves in the market if that were the case. You would have extraordinarily productive teams using these languages -- along the lines of what PG hypothesized 15+ years ago.
There's K and Kdb, et. al. (I think they're on to "Q" now.)
I'd argue very little of what makes K and Kdb special has to do with the language per se, but with the focus on array operations, and with the particular runtime environment of Kdb (heavily focused on distributed services that logs (and can replay) operations against a columnar database with most operations in-memory).
Q to me is a tacit admission that the obtuse syntax of K is not necessary to get the benefits of K, and that K presents too steep a learning curve for most.
They're fascinating, and I think vastly underrated, but they're hard to grasp even without the syntax hurdle...
1. The terse language is actually quite expressive and very well suited to vector programming. So less code is more. Also gets you thinking in a particular way...
2. The database, a mix of transactional, in memory, and fast analytics (aggregations, joins..) concepts
3. The runtime framework: Kdb is an ideal building block for a network distributed system. Opening sockets and connecting to other Kdb instances is simple.
2/3 I buy, but for 1 I think you need to decouple the syntax from the underlying array-programming model. K-level terseness and the semantic structure that makes K well suited to vector programming are pretty much entirely orthogonal.
You can implement the core of K's array manipulation in other languages and ends up with similarly terse code if that's what you want. But you can also produce something that will be far more readable to most people without losing the expressiveness by e.g. opting for naming some of the operations etc.
> I'd argue very little of what makes K and Kdb special has to do with
the language per se, but with the focus on array operations, and with the
particular runtime environment of Kdb
I'm also not convinced that K as a language is essential to kdb as an
application but it's undeniable that they have "proven themselves in the
market", eh? It's a niche, but a well-paid one, and they don't have any
competitors (with e.g. easier syntax.)
Are you familiar with arcfide's Co-dfns compiler?
https://github.com/Co-dfns/Co-dfns I ask because it might serve as a
counter-example of a "killer app" that relies on the concision of the
syntax.
> I think you need to decouple the syntax from the underlying
array-programming model. K-level terseness and the semantic structure
that makes K well suited to vector programming are pretty much entirely
orthogonal.
FWIW, I think so too. You could say that things like Numpy and Pandas are
moving towards integrated array processing, eh?
They have proven themselves in the market, but it's impossible to decouple the language agnostic benefits of KDB from K's syntax in that respect. That said, they have plenty of competitors - just most of them are just run of the mill languages.
But it can't easily (and perhaps essentially) be decoupled and still be usable.
At first, the math notation for "x ^ n" is cryptic, and writing "x * x * ... * x repeated n times" is clearer. But then, you grok exponentation, and everything longer than "x^n" becomes too verbose and needlessly complex; furthermore, once you've grokked it, "x^n" naturally extends to negative, rational and real n naturally, whereas the expanded form does not.
K is similar; At first, it seems needlessly terse, weird and opaque. But once you've grokked it, it's clear that it's much more than the underlying array model (which, as you pointed out, can be and has been implemented in other environments).
As supporting evidence, I bring numpy; It has the same kind of array operations, and as-high-a-level of language; and yet, code in numpy written by most people is still an order of magnitude longer (and slower) than code produced by people writing K. There is a selection bias here (Everyone uses python, only K compatible people use K), but I think it's more.
Look at an example from [0] - the complete function (executable by the 300KB entire K runtime, independent of any library, and with named arguments which makes it 50% longer than some people would write it):
{[t;f;o]{x y z x}/[_n;o;t f]}
This is a K idiom known afaik as "converging multicolumn select"[1]. "t" is a table; "f" is a [list of] columns, e.g. `price`size ; "o" is a [list-of] subset-and-ordering functions, one for each column.
To collect all the rows from table "sales", where time is more than 9am, price is less than 100, and ordered by decreasing size, you would call it with the arguments [sales;`time`price`size;9h00<,100>,>:]
Although it is possible to write such a simple, generic, useful idiom in Python/numpy, I have personally never seen it done. R's data.table (and its Python port) come close in a limited subset of cases, but the extreme usefulness of converging multicolumn select does not extend; also notice that the call has exactly the specification, no boilerplate or even keyword (and just minimal punctuation). This example is not unique. e.g. "flatten" is ,// ; Yes, that's the entire function implementatiion, and no, it's not special cased.
K optimizes the use of your eyes and brain as a pattern matching device. It has been my experience that, much like good math notation (e.g. Leibniz's differential notation), good programming notation makes things easier and evident in a way that is not captured by equivalent semantics that aren't human compatible in the same sense.
> But it can't easily (and perhaps essentially) be decoupled and still be usable.
I'd argue it's not usable to most people in its current form at all, which is why they remain tiny niche languages.
> At first, the math notation for "x ^ n" is cryptic, and writing "x * x * ... * x repeated n times" is clearer. But then, you grok exponentation, and everything longer than "x^n" becomes too verbose and needlessly complex;
This is contrived. The argument is not for writing out like that. Now of course x^n is well understood by most, but many languages do offer longer versions, such as e.g. math.pow(x,n). For x^n I don't think that is necessary, but for less common operations I'd much rather have a word that hints at a concept than a symbol that does not.
For most us remembering a large number of single-character operators slows us down, rather than speed us up.
> Although it is possible to write such a simple, generic, useful idiom in Python/numpy, I have personally never seen it done.
I don't use either, but e.g. considering Ruby, the main reason why you wouldn't tends to be the lack of the equivalent of a "table" type. With a table type that provides suitable methods, such as e.g. provided by typical Ruby ORMs, you might end up with something like this:
Of course we wouldn't write it like that, because it is unreadable.
I think this is typical of an attitude difference. I don't see the value in your example, because to me it is semantic overhead that requires me to slow down when reading code.
> also notice that the call has exactly the specification, no boilerplate or even keyword (and just minimal punctuation).
This is part of the negative aspect of K syntax to me, not a positive.
> K optimizes the use of your eyes and brain as a pattern matching device. It has been my experience that, much like good math notation (e.g. Leibniz's differential notation), good programming notation makes things easier and evident in a way that is not captured by equivalent semantics that aren't human compatible in the same sense.
My experience is that there is a sharp dividing line between people who are happy reading maths, who tends to be ok with syntax like that, and people who rely on visual layout to recognize code who tends to absolutely hate it, and judging by which programming languages are successful, the latter group seems to be substantially bigger. I'm glad you're happy with a syntax like that.
But to me it is not useful, because I can't skim it and quickly get an idea what it does.
There's K and Kdb, et. al. (I think they're on to "Q" now.)