One of the things I've tried to do differently in Arc is to design the language based on what actually happens in programs, rather than a priori theories. For example, let is for the one-variable case because I looked at a lot of CL code and found 76% of the lets bound one variable. Cond doesn't have implicit progn because I found most cond clauses didn't use it. Etc x 100.
So in reply to the a priori argument that shadowing global function names will cause problems (How big? I don't know. Big!) I reply that extensive experience has shown that it's just not a big deal.
I'll add modules eventually, of course. But it's not a high priority.
I didn't say it was a big problem, I said it was a real problem. And the larger your code base, and the more people start sharing code, the more real it will be. It is only a matter of time before someone writes something like:
(let foo snoz (jims-macro frotz))
and gets an error of the form "42 is not a valid function" and has to spend an hour puzzling over where that error came from before realizing that Jim's macro expands into Bob's macro which expands into Ann's macro which uses a helper function called foo.
Is it a "big" problem? Maybe not. But it's a real problem, and given that one of your stated goals was to design "the 100 year language" I think it's a shame that you chose to solve it by placing the burden on the programmers. (And I use the plural advisedly here, because the potential for encountering this problem grows as the community grows and people start re-using each other's code.)
But I would reiterate that the lack of abstract data types is IMO a much bigger problem than the lack of hygienic macros. (And just so you know I'm not just sniping from the sidelines, I have proposed solutions implemented in CL which I'd be happy to help port to Arc if you agree that this is in fact a problem.)
I'm not a lisp programmer neither I know the peculiarities of each lisp implementation, but I think I understand it a little bit.
My question is, what hinders Arc (or other Lisp if it has the same problem) to create an operator that finds any global symbol from the top, like when an absolute path starts with "/" to find a file?
For instance, let's stay that there is an "unshadowable" operator called "." and let's take your example:
(let foo snoz (jims-macro frotz))
Well, that shadowed "foo" is causing the problem. As you said ("Jim's macro expands into Bob's macro which expands into Ann's macro which uses a helper function called foo."), this symbol is shadowing the actual function "foo" used by Ann's macro.
However, if Ann's macro were written carefully (and all macro writes should know that macros can be used in any context, any scope), Ann would care to call "foo" from the top using the "." operator, like:
...
(. foo args) ; excerpt of Ann's macro.
...
In this case, in every "(let)" or another scope definition method, you could always escape to the global scope using this operator, behaving like a hole to the outside world.
Of course, I'm just a Lisp newbie, maybe I'm just creating a huge mess with the concepts :-).
That's kinda the right idea. There are two problems with your specific proposal: 1) it places a considerable burden on the programmer to annotate every invocation of a top-level function. For complex macros that can become quite annoying. 2) Baking the concept of a single top-level into the macro system makes it hard to add modules later.
If you pursue the idea of having the system walk the code and automatically insert those annotations you will (almost certainly) end up re-inventing hygienic macros.
(For the record, I much prefer the Lisp2 solution, because it's the hacker's solution. It's not mathematically elegant, but it's simple (compared to hygiene) and it gets the job done.)
How did you arrive at these conclusions? Did you use some programmatic analysis of CL source code to report these stats? Which codebases were your reference?
Yes. I used all the CL and Scheme source of my own that I could find, and I also asked Ken Anderson to analyze some big collection of CL (I think it was) code at BBN.
I'm sure I'm not a good representative Lisping example, but I routinely use let to bind more than one variable, especially when extracting program state from a URL in multiple variables, and probably use the implicit progn in cond 70% of the time. I can see the value of not allowing the implicit progn which would force creating more numerous and smaller functions, but in the case where I have only 2 expressions in a cond, a function just for that would seem a bit verbose. But again, this is really no big deal as with ARC, one can write a macro utility to put the multiple option back in.
Nice article Ron, non-pc & certainly learnt something reading it. Though I reckon this ones a ripper...
"... So I can't really go into many specifics about what happened at Google because of confidentiality, but the upshot was this: I saw, pretty much for the first time in my life, people being as productive and more in other languages as I was in Lisp. What's more, once I got knocked off my high horse (they had to knock me more than once -- if anyone from Google is reading this, I'm sorry) and actually bothered to really study some of these other languges I found #myself# suddenly becoming more productive in
other languages than I was in Lisp. For example, my language of choice for doing Web development now is Python. ..."
"For example, my language of choice for doing Web development now is Python. ..."
To Lisper:
It seems several people are comparing Arc to other Lisps.
I think that's somewhat unproductive. The Lisp user base is so small now that there isn't much sense "marketing" to that audience by comparing Arc to Scheme, CL, etc.
So, an alternative question: what would it take for you to use Arc instead of Python for web programming?
> It seems several people are comparing Arc to other Lisps.
Not me (at least I hope not). I was trying to judge Arc entirely by the standards that Paul set for it in his foundational essays.
> what would it take for you to use Arc instead of Python for web programming?
At the very least better documentation :-) There appear to be some really cool things in the Arc web infrastructure, but the only way to figure out how to use them at the moment seems to be to reverse-engineer the code.
I agree. I was alarmed, too, by non-hygienic macros combined to the terse names of standard functions. It seems too easy to shadow them (I have to change my custom to name the argument 'obj').
About generics, I expected that the type annotation system (annotate, type, rep) would be the basis of some sort of user-expandable generic dispatching mechanism. In arc0 only macros using it and its handling is built-in to the core, so it's not clear how PG want it to evolve. He's probably putting it off, but it's where the fun is...
Another point I was alarmed is the automatic promotion of non-function objects to a function that tests equality to it (some, all, rem, keep, ... all does that via testify). It is useful since it captures typical pattern. However, on the other hand, aggregate data is 'callable' when used in the function position. I can make a table that maps elements to booleans and I can use it as if it is a predicate, but that illusion breaks once I pass it to other procedures which eventually pass it to 'some'. (In my Scheme implementation, any objects can be made callable. It is handy, but I then learned that to keep that "handiness", I had to give up most of checking if something is procedure or not.)
Look I know it's an ignorant post of mine. But I use CLISP and I am very busy with a lot of things these days. Too many to count...
But my plan is to continue to use CLISP until I become convinced that arc will help me in some way. I have almost 20,000 lines of CLISP I have developed and thus I am quite reluctant to even contemplate a switch.
I will put it bluntly, "what's wrong with common LISP?"
If somehow people don't like all the redundancy, well use the subset you like. That's what I do. Sure it would be nice to have a language that forced some kind of uniformity and consistency in its nature. But these are abstract/fluffy terms. And like I said I have 20k lines of LISP that I am not interested in porting...
But if a crystal clear reason were to emerge, then yes I would do it. I do have an open mind.
>I will put it bluntly, "what's wrong with common LISP?"
PG already covers many of his objections to CL his essays. I will not repeat them here. It might be more fair to ask "What's wrong with Scheme?", as Arc seems much closer to Scheme than CL.
As I see it, one of the biggest reasons to create Arc is social. It doesn't have to do anything CL and Scheme don't already if PG and RTM can build a real community around it.
Yes, sorry I was being cranky yesterday. I had just come off a harrowing (but productive) week on the Job... Publish or perish time for me!
So after my hangover wore off (we went out last night), I decided that I shouldn't be such a grouch and I should take a gander at arc rather than just shoot my mouth off about how I don't want to dick around with my near 20k lines of CLISP code (wc #of lines, which include comments).
I looked very quickly at the tutorial and I have to say that I very much like the feel. And although I am under strict personal orders not to think about code these days, I can't help but to plot out a potential porting path. The first thing will be to build some macros so that my common lisp code looks more arc like. This would force me to clean a lot of things too, perhaps knocking it down to 10k if I am lucky. I also know I am doing a lot of dumb things here and there so perhaps if I am really lucky/skilled I can get it down to 5k.
Then I will see how things evolve and do the port when I get the time. It looks like arc will have good momentum and will breath life into the community, so yes I think it's a great thing.
So in reply to the a priori argument that shadowing global function names will cause problems (How big? I don't know. Big!) I reply that extensive experience has shown that it's just not a big deal.
I'll add modules eventually, of course. But it's not a high priority.