It's about as fun and easy to implement a simple thing that looks like a forth as it is to implement a simple thing that looks like a lisp. But that's where the ease of forth ends for me.
I find forth basically impossible to read even when, at best, authors leave a comment on basically every 1-2 words. Writing it is pretty hard too.
In contrast it's about as easy to be productive in a lisp (CL or Scheme) as it is in Python or JavaScript (give or take useful 3rd party libraries).
Even if they're from the same time period and are interesting for being not very Algol-like, I don't think it makes a lot of sense to discuss Forth and Lisp in the same conversation. They were both improvements on very different things.
Also, when we talk about reading code, the reading that matter is this:
- how easily can you read incorrect code and find the problem.
- how easily can you convince yourself that correct code is correct.
x86 assembly language is highly readable: e.g. it's completely unambiguous
that "xor eax, eax" clears the eax register, independently of the previous
or following line of code.
An x86 assembler can tell you that "xor eax" is missing an operand.
In Forth, a word can receive insufficient arguments due to something distant
in the program, not related to that word.
For instance, suppose we push 200 operands onto the stack (o1 o2 o3, ...), followed by words which are supposed to decimate those operands down to one value. Then suppose we remove some operand in the middle, like o17. There will necessarily be a stack underflow. But that underflow will not happen inside that word which operates on o17. That word will cheerfully use o18 as its argument, thinking that it's o17. It is the word which operates on o1 that will bomb; at that spot in the execution when the correct code would pull o1 from the stack, the stack will be empty.
In a language with syntax, if we edit an expression tree to randomly remove an operand, such that a function's arity is no longer satisfied, the error will be pin-pointed to that function call, possibly at compile time. Before even running the program your editor can be made to jump there to fix the error.
(In Lisp, there is a vaguely analogous problem at read time with parentheses. If you have to close 25 parentheses, but close only 24, it is ambiguous which one of the 25 open parentheses is missing the closing one. The situation is pretty trivial: all parentheses are the same, so it doesn't matter. The reader can just report the location of the leftmost one of those 25 as the likely start of the issue.)
Good programming in Forth is all about scope. As soon as you make a word do too much you will end up with all of the problems you listed. But if you keep your scope down, not for elegance but simply to avoid cognitive overload and the potential for errors then you can get very far before any of this becomes an issue, and the amount of bang for the buck is very large.
Forth programs tend to have a pretty distinct style, closely related to whoever wrote them. Some see this as an advantage, for others it is a huge drawback because they can't or won't put themselves into a mental model that is non-obvious at first reading. This is however not a defect, it is pretty much the intention: Forth is a language to create DSLs, the words you define make up that new language and that is the language that you will ultimately solve your problem in. I haven't done any serious Forth coding in over two decades but when I did I loved working with it. But if I had to do it today my gripes would not be with the language as such but with how hard it would be to make a Forth project into a team effort.
Forth is an artisans tool, it is for making jewelry, not for making apartment buildings.
Indeed. I love using Forth for focused tasks, like building a game in a game jam, or building chat bots. I've even collaborated with a person in a Forth game and it worked out fine because we were spending a lot of time talking about our game and working through words and gameplay. Scaling this to an app developed by a large team would be a nightmare.
(As an aside, I feel that this is what attracted the CollapseOS author to pursue Forth. Most of the folks interested in the retro collapse computing seem to really like creating artisinal/small team programs rather than large collaborations.)
> how hard it would be to make a Forth project into a team effort.
A formal review process for merging any commit, requiring approvals from several developers; two or three experts on good Forth code practices the team; and a document outlining what those are --- ought to do it? I think.
Plus tooling. Forth can undoubtedly be linted for issues like definitions without stack effect annotations, so the reviewers don't have to waste time pointing that out.
You are going to find two or three experts on good Forth code practices on the same team?
I wouldn't even be able to find one, and likely, if I did their idea of what 'good Forth code practices' are would be a world away from mine. And in a way that is what makes Forth as interesting as it is, it's like that vest with a hundred pockets that has everything there just the way you like it. But for someone else that vest would not work at all, simply because they are not you.
If you're going to plunge into team Forth development, you probably should somehow scramble to have some experts, right? If you can't find them, make it the mission of some existing devs to become experts. If you have people who are experts in anything else, including the fact that they are experts, those would be good candidates.
I've previously posted some stuff about Mitch Bradley -- I have used various versions of his ForthMacs / CForth / OpenFirmware systems, and I was his summer intern at Sun in '87!
Mitch is an EXTREMELY productive FORTH programmer! He explains that FORTH is a "Glass Box": you just have to memorize its relatively simple set of standard words, and then you can have a complete understanding and full visibility into exactly how every part of the system works: there is no mysterious "magic", you can grok and extend every part of the system all the way down to the metal. It's especially nice when you have a good decompiler / dissassembler ("SEE") like ForthMacs, CForth, and OpenFirmware do.
>DonHopkins on Mar 26, 2015 | parent | favorite | on: The Elusive Universal Web Bytecode
>The other "universal bytecode" that came out of Sun and actually has seen a lot of use is Mitch Bradley's Open Firmware Forth based boot ROMs, used by Suns' 68k, SPARC and x86 boxes, Apple's PPC Macs, IBM, ARM, OLCP XO-1, and many other systems.
DonHopkins on Aug 7, 2016 | parent | favorite | on: “EFI? Intel has been trying to shove that down ou...
>Mitch Bradley originally developed a Forth system at Sun for use diagnosing and developing hardware, by burning it into ROM and running it via a serial port.
>It was based on Langston and Perry's Forth-83, and had a meta compiler that could target different word sizes and architectures. He made it even more architecture and word size independent, implemented interactive top level loops and conditionals, emacs-like line editing, all kinds of low level device drivers and testers that ran in stand-alone mode, and many other features, including full 16 and 32 bit support with a vocabulary for writing word size and endian independent code.
>He ported Sun Forth to 68K and SPARC Sun workstations, as well as the Amiga and other systems. It ran in both stand-alone mode (from disk, tftp or ROM), or under Unix. Under Unix, it could dynamically relocate and link in Unix libraries, and you could call back and forth between Forth and C.
>Sun Forth eventually evolved and standardized into the Open Firmware [1], whose purpose was to support machine independent byte code [2], so plug-in hardware cards could include ROMs with Forth byte code drivers that ran on 68K, SPARC, x86 and other systems.
>Sun shipped it with the SPARC workstations, Apple adopted it and shipped it on their PowerPC Macs, IBM shipped with their POWER servers, and Mitch worked directly with the OLPC project extending OpenFirmware to support the OLPC XO-1 Children's Computer secure and power efficient hardware. [3]
>>OLPC Wiki: Open Firmware
>>Open Firmware is the hardware-independent firmware (computer software which loads the operating system) that the XO runs.
>>It was developed by Mitch Bradley at Sun Microsystems, and used in post-NuBus PowerPC-based Apple Macintosh computers (though it has been dropped with Apple's transition to Intel processors), Sun Microsystems SPARC based workstations and servers, IBM POWER systems, and PegasosPPC systems, among others. On those computers, Open Firmware fulfills the same tasks as BIOS does on PC computers.
>>For example Fedora and Debian use the YaBoot BootLoader for Open Firmware.
>>The Open Firmware user interface includes a FORTH-based shell interface. FORTH is a powerful high level language that is remarkably compact. A complete Forth development environment including compiler, decompiler, assembler, disassembler, source level debugger, and assembly language debugger is present in the XO boot ROM (SPI FLASH). With the Open Firmware Forth system, you can directly access all of the hardware devices on the XO, use built-in functions like selftest diagnostics and games, and even write complete applications, without needing any external tools. The bulk of Open Firmware is written in Forth, so the source level debugger can be used to debug Open Firmware itself.
>Emacs qualifies as a window manager in my book! ;)
>That code you linked to is from Mitch Bradley's "Forthmacs", which ran on Sun workstations including 68k i86 and SPARC, and also Atari ST, Mac and other systems. He developed it into the "Open Boot ROM" architecture, which was used in Sun workstations and Apple PowerPC Macs as well as the OLPC children's laptop.
>On SunOS, Forthmacs had a library clink.f with the ability to dynamically relocate and link Unix libraries so that you could call them from Forthmacs, pass arguments on the stack, etc. SunOS didn't actually support shared libraries or dynamic address relocating at that time, so Forthmacs simply ran the Unix linker utility to create a file with the library relocated to the desired address space in the FORTH dictionary, and then read that file into memory, define its symbols in the FORTH dictionary, and let you access its variables and call its functions directly from FORTH!
>That's how Mitch originally integrated MicroEmacs with Forth to make Forthmacs, and how I later integrated "uwm" into FORTH: I refactored uwm so instead of having an event loop in the main function, it was a library that could be called by FORTH, which would link the library in and run the main loop itself, calling into the library as needed to initialize and handle specific events (_uwm_init, _uwm_poop).
>DonHopkins on July 7, 2018 | parent | favorite | on: Tali Forth 2 for the 6502
>Cool! Does it include a FORTH 6502 assembler written in FORTH? I love writing assembly code in RPN with Forth macros!
>You can write FORTH code with loops and conditionals and any kind of logic and parameters, that dynamically assembles machine code! Much better than your typical macro assembler.
>Here's some 6502 assembler for an Apple ][ SUPDUP terminal emulator that does ram card bank switching:
>Mitch Bradley's 68k forth lets you use interactive conditionals and loops in the top level outer interpreter!
>I think he wrote a paper about that feature, which I might be able to dig up ... Here's something related he wrote about refactoring the outer interpreter, but not what I'm thinking of:
I was also an "intern" of sorts for Mitch as recently as 2009 or so. I wrote the audio driver for OLPC XO 1.5 and went to Taipei to help with the bringup. I was amazed to watch Mitch using Forth to interactively tweak the DDR memory settings, find bugs in the motherboard, and find bugs in the CPU which was a godawful x86 from VIA that had seemingly only been debugged enough to boot windows :)
DonHopkins on Dec 18, 2019 | parent [–]
That must have been a fun and unique opportunity! His programming and debugging process has to be seen to be believed. OpenFirmware is definitely a highly polished work of love. Its well-commented and meticulously organized source code reads like fine literature or classical music! Stuff like the metacompiler and kernel, for example:
FORTH doesn't have an IFELSE unless you define it yourself of course, but it would have no way of telling which part of your code was the conditional, part to do if true, or part to do if false, since white space and line breaks have no meaning to it, except for a few cases like \ comments. The IF comes between the conditional and the part to do if true, the ELSE comes between the part to do if true and the part to do if false, and the THEN comes at the end.
The structure of PostScript code, conditionals, and flow control is homoiconic, made of nested arrays, like polymorphic JSON arrays, where FORTH code is flat untyped arrays of threaded pointers with relative branching.
Forth has an "inner interpreter" called NEXT that threads from one pointer to the next (sometimes implemented as one machine language instruction, or just a few), and also an "outer interpreter" or compiler that translates text into threaded code for the inner interpreter.
Here's Mitch Bradley's implementation of interactive control structures (IF, ELSE, THEN, BEGIN, UNTIL, AGAIN, REPEAT, WHILE, DO, LOOP, etc) in the outer interpreter of his FORTH system, which actually allows you to use conditionals and loops at the interactive top level of the interpreter, not just in compiled words (which solves a traditional and frustrating limitation of classic FORTH).
Here's some more of Mitch's beautiful code from OpenFirmware, his Forth kernel meta-compiler written in Forth, which supports 8, 16, 32, an 64 bit, big-endian and little-endian architectures, as well as direct, indirect, and token threaded code:
wow, thanks for the explanation! I was not aware of these subtle differences between the forth and posctript styles. Having implemented a few stack-based mini-languages for mathematical expressions, I always pick a "greedy" evaluation style, where "ifelse" is a function of three arguments (pops the top three values of the stack) which have been evaluated beforehand. Since there are no side effects, the end result is the same, but it is certainly not a "true" ifelse, since both sides get evaluated regardless of the condition.
I can't find any reference to this in ANS Forth 1994, other than appendix C. 3 Hardware Implementations of Forth which says: "In the mid-1980’s Zilog developed the z8800 (Super8) which offered ENTER (nest), EXIT (unnest) and NEXT in microcode." (which is interesting, by the way).
> Every FORTH primitive that we write has to be ended by NEXT. Think of it kind of like a return.
Interesting; it's "like a return" because it's a form of continuation: a tail call to the next thing.
I wonder if this could be done in C with function pointers? If we end a function like this:
typedef void (*fptr_t)(); // old-style, on purpose
void primitive(fptr_t *ppSelf) // pointer into array of fptr_t's
{
// ... perform primitive
(*++ppSelf)(); // increment to NEXT primitive and call
}
Will modern C compilers turn this indirect call in tail position into a tail call, like a direct call.
In C you'd try to get an indirect jmp into the body of the low level compiled version of the word. So likely you'd just use an 'asm' instruction and ignore the stackframe that C normally sets up because that would be overhead you do not need.
Alternatively, you could implement your own data stack and use the regular stack for control flow, but this would incur a performance penalty.
See those () at the end of your 'next' are exactly what you don't want, they will mess up the stack. Now you have to start the definition of every word with something that eats up that return address and the stack frame set up at the beginning of the word definition again. You don't really need either of those. But you do need a data stack.
The inner interpreter of Mitch Bradley's portable and efficient CForth is simply implemented with a big switch statement, in the function called "inner_interpreter".
The C compiler typically compiles that code with the big switch statement into an efficient jump table, behind the scenes.
Properly speaking, that's a token threaded interpreter, since it switches on token numbers in the code field, instead of indirecting function through pointers to the C equivalent of "code" words.
Compiled user defined Forth colon definitions, <builds and does> definitions, etc, do thread through code field address pointers though -- see the default case of the switch statement:
>DonHopkins on Jan 9, 2015 | parent | context | favorite | on: Design of Lisp-Based Processors Or, LAMBDA: The Ul...
>The Novix FORTH chip was a pretty cool implementation of FORTH in hardware -- it had separate data and return stacks, which it could push or pop at the same time, so the compiler could combine several FORTH words into one opcode.
>The Novix NC4016, formerly called the NC4000, is a 16-bit stack based microprocessor designed to execute primitives of the Forth programming language. It was the first single-chip Forth computer to be built, and originated many of the features found on subsequent designs. Intended applications are real time control and high speed execution of the Forth language for general purpose programming.
>The NC4016 uses dedicated off-chip stack memories for the Data Stack and the Return Stack. Since three separate groups of pins connect the two stacks and the RAM data bus to the NC4016, it can execute most instructions in a single clock cycle.
>The NC4000P is a single chip FORTH Engine based upon minimum hardware concepts developed by Mr. Charles H. Moore. This highly parallel machine architecture directly executes FORTH primitives in a single clock cycle. The initial implementation of this device is based upon a 4000 gate CMOS semicustom integrated circuit operating at an 8 MHz clock rate.
But I'm preaching to the choir, Jacques! ;) Dadadatablblblblbl!
>jacquesm on April 8, 2012 | parent | context | favorite | on: A Forth Story...
>Quite a story, and it seems it does not have a happy ending, but for all the wrong reasons.
About 2/3rds in (past the 'read more') the Novix rates a mention. I think that that is possibly the most underrated CPU design that ever saw the light of day, at the time it was so out of the ordinary that only very few people knew what to do with it.
>A high level language (forth is by most definitions a high level language) directly executed by the CPU.
>At the time I worked for a company called dadadata in the Netherlands, they were in the process of developing a bunch of real time software to process video images for classification and recognition (1988 or thereabouts). The Novix chip arrived and would have definitely blown the socks of anything that I could have done with a regular PC at the time, if not for one small problem: someone dropped a dime into the power supply and the whole development kit was toast. A couple of all nighters later we had a working reproduction of the code in 'C' and that was used for the customer demonstrations. Still, the power of that little chip is something I'll never forget and it is a pity that the Novix and its successor which iirc was called shboom never made real headway. The forth code for the novix was probably only about 10% or so in size of the equivalent C code. The head forth honcho there called C sneeringly a 'great' language.
>There is still hope, even today that one day that power will finally be available in some package that sees wide distribution, the heritage of Novix lives on in the greenarrays line (which is now shipping dev boards and chips).
I have a hunch that by using GNU C computed labels (extension not in ISO C), you could have an actual NEXT macro which increments the pointer through the label table, and does a goto through the incremented value.
DonHopkins 6 months ago | parent | context | favorite | on: Moving Forth (1993)
>Some FORTH systems even have a "metacompiler" that lets one FORTH system compile another FORTH system for the same or different CPU, word size, byte order, threading technique, with or without a built-in compiler, etc, from the same source code!
>OpenFirmware (the FORTH burnt into boot roms of SPARC, PowerPC, OLPC, and other systems) is a great highly refined example that supports many different architectures:
The OpenFirmware kernel is a Forth meta-compiler, which can compile itself on any architecture, and also cross-compile for different target architectures.
It has cross-architecture extensions to FORTH (like \16 \32 comments and /n /n* generically typed words) that make it possible to write platform, word size, byte order, and threading independent code, and compile images (stripped or with headers) for embedded systems (like the OLPC boot ROMs) and new CPU architectures (like Sun's transition from 68K to SPARC), and share code with a more powerful development environments.
>Many advocates of the language Forth call the process of creating a new implementation of Forth a meta-compilation and that it constitutes a metacompiler. The Forth definition of metacompiler is:
>"A metacompiler is a compiler which processes its own source code, resulting in an executable version of itself."
>This Forth use of the term metacompiler is disputed in mainstream computer science. See Forth (programming language) and History of compiler construction. The actual Forth process of compiling itself is a combination of a Forth being a self-hosting extensible programming language and sometimes cross compilation, long established terminology in computer science. Metacompilers are a general compiler writing system. Besides the Forth metacompiler concept being indistinguishable from self-hosting and extensible language. The actual process acts at a lower level defining a minimum subset of forth words, that can be used to define additional forth words, A full Forth implementation can then be defined from the base set. This sounds like a bootstrap process. The problem is that almost every general purpose language compiler also fits the Forth metacompiler description.
>When (self-hosting compiler) X processes its own source code, resulting in an executable version of itself, X is a metacompiler.
>Just replace X with any common language, C, C++, Pascal, COBOL, Fortran, Ada, Modula-2, etc. And X would be a metacompiler according to the Forth usage of metacompiler. A metacompiler operates at an abstraction level above the compiler it compiles. It only operates at the same (self-hosting compiler) level when compiling itself. One has to see the problem with this definition of metacompiler. It can be applied to most any language.
>However, on examining the concept of programming in Forth, adding new words to the dictionary, extending the language in this way is metaprogramming. It is this metaprogramming in Forth that makes it a metacompiler.
>Programming in Forth is adding new words to the language. Changing the language in this way is metaprogramming. Forth is a metacompiler because Forth is a language specifically designed for metaprogramming. Programming in Forth is extending Forth adding words to the Forth vocabulary creates a new Forth dialect. Forth is a specialized metacompiler for Forth language dialects.
Mitch's OpenFirmware code is one end of the readability spectrum. But for shock value, here are some beautiful but inscrutable FORTH code excerpts that define cellular automata rules with lots of deeply nested conditionals around mysterious but exquisitely indented BEEPS and BOOPS and line noise that makes Perl look clean and elegant. ;)
The specific issue with reading Forth is that unlike other languages, you, the reader, need to be intimately familiar with the call semantics of every word that you're trying to read. In most other languages, this is not a problem as (most) are explicit in what's being passed and returned. And Forth is unlike BASIC where most most everything is implicitly global.
This issue with Forth is because you have the "invisible" stack as a first class parameter passing system. So, given a string of Forth words, while the stack may be apparent at the start (perhaps as a comment at the entry of a high level word), what happens to that stack during processing is opaque to the reader. Then you have have issue within individual words of stack gymnastics to get everything lined up for the interior processing, or readying to call something else. Now, if the reader knows each word and its behavior, then they can follow along but under the cognitive load of effectively "executing" each word they see to visualize its stack impact. You can learn to skip a bunch of the gymnastics because, assuming it works, you "don't care", it's just the noise of processing "knowing" it's converting from X to Y, and just knowing the Word W needs Y, how it gets Y is, often, less important. But it can impact the logic standing out buried in the manipulations to invoke everything properly. In Lisp, for example, I do a lot of "converting" in the (let ...) section, then the code works on the results of that. If you "ignore" the (let ...) section, the code is usually reasonably clear. That tends to be difficult in classic Forth as that gets all intermixed.
It's one thing when you've built up your code base over time, and are familiar and intimate with it incrementally. But it's quite another approaching a foreign code base.
Over time, familiarity reigns and lowers the impact, but on an initial read it can be quite challenging.
It's also fair that at some point, at the higher levels of abstraction, all of that goes away, and the code just "reads", but, inevitably there's a bunch of sausage makings supporting it that you need to make sense of.
> is that unlike other languages, you, the reader, need to be intimately familiar with the call semantics of every word that you're trying to read
Not really "unlike other languages". You can hardly read any language if you don't know what the words mean.
In Forth, even if you know what every word means and how many operands it consumes and produces, it's not obvious what in the program produces the operands and what consumes them. There is no syntactic enclosure or other such clue which links them together.
Forth could be like
getangle ... hundreds of words ... cos
where getangle produces the operand that is consumed by cos, hundreds of words away.
Nothing links these two locations together visibly. No syntactic enclosure, no def/ref of a shared identifier.
Probably, the answer in Forth programming is: don't do that. Don't have code that has hundreds of words in one definition. If no word definition consists of a large number of items, then you minimize this problem.
In "mainstream" languages, though, we can have a large body of code such as a big function that is hundreds of lines long, and it can be reasonably easy to deal with if it is otherwise well structured. If a variable is defined at line 50, which is then used at line 550, you can fairly easily track that in your editor. Not as nicely as 50 verus 65, but the linkage is visible enough.
That also gave programmers good incentives to write decent text editors and file systems that could deal with any number of lines, to get around that historic annoying limitation of FORTH. ;)
Granted it was nice to have a dead simple file system for embedded devices, but forcing you to keep your word definitions short always seemed like a lazy excuse for not having a real text editor and file system.
Sometimes you don't, but don't blame that on trying to incentivize programmer behavior. FORTH hardly ever "forces" programmers to do anything, so that excuse never rang true to me.
The "block" approach also forced you to not write comprehensive stack comments and documentation too, which is more important than keeping your word definitions short.
Otherwise you get dense sparsely documented code like this (which came from my Apple ][ Forth 40x24 screens):
Here's some IBM-PC Forth for the CAM-6 cellular automata machine that was block based -- check out the unique idiosyncratic right-justified reverse-indentation style (starting with KGET), which is not standard Forth style, but sure looks cool and poetic, like E E CUMMINGS ON CAPS LOCK:
DonHopkins on May 3, 2020 | parent | context | favorite | on: History of Logo
FORTH is the ultimate macro-assembler!
The assembler is just written in and integrated with FORTH, so you have the full power of the FORTH language to write macros and procedural code generators!
And it makes it really easy to call back and forth ;) between FORTH and machine code, with convenient word definitions for accessing the FORTH interpreter state.
Here's part of my SUPDUP terminal emulator for the Apple ][ with some 6502 code for saving and restoring lines of text in a bank-switched memory expansion card:
Here is a great example of FORTH and 8086 assembly code for hardware control (written by Toffoli and Margolus for controlling their CAM-6 cellular automata machine hardware), starting with "CAM driver routines" and also "creates fast code words for picking out bits of variable X":
>Starting to write programs for the CAM-6 took a little bit of time because the language it uses is Forth. This is an offbeat computer language that uses reverse Polish notation. Once you get used to it, Forth is very clean and nice, but it makes you worry about things you shouldn't really have to worry about. But, hey, if I needed to know Forth to see cellular automata, then by God I'd know Forth. I picked it up fast and spent the next four or five months hacking the CAM-6.
>The big turning point came in October, when I was invited to Hackers 3.0, the 1987 edition of the great annual Hackers' conference held at a camp near Saratoga, CA. I got invited thanks to James Blinn, a graphics wizard who also happens to be a fan of my science fiction books. As a relative novice to computing, I felt a little diffident showing up at Hackers, but everyone there was really nice. It was like, “Come on in! The more the merrier! We're having fun, yeeeeee-haw!”
>I brought my AT along with the CAM-6 in it, and did demos all night long. People were blown away by the images, though not too many of them sounded like they were ready to a) cough up $1500, b) beg Systems Concepts for delivery, and c) learn Forth in order to use a CAM-6 themselves. A bunch of the hackers made me take the board out of my computer and let them look at it. Not knowing too much about hardware, I'd imagined all along that the CAM-6 had some special processors on it. But the hackers informed me that all it really had was a few latches and a lot of fast RAM memory chips.
Don, I don't know anything about you that I haven't seen on Wikipedia, but you're evidently extraordinarily intelligent, erudite, and experienced, and bring a phenomenal amount of knowledge to conversation.
I'm in general skeptical of inquiries into people's personal methods, but in your case I just have to ask: do you use some kind of system for keeping your quotes, excerpts, and data to hand for these kinds of threads?
Thank you for your kind words, and for asking a great question!
My secret system that has gotten me through the coronavirus pandemic is simply an investment in an automatic coffee machine that grind beans and foams milk for me. ;)
I use HN search and google site search, copy and paste and clean up old postings, and then check through all the links (since HN abbreviates long links with "..." so I have to copy and paste the full links manually), and I update the broken and walled links with Internet Archive links.
I realize some of my posts get pretty long, and I apologize if that overwhelms some people, but it's a double edged sword. One important goal is to save other people's time and effort, since there are many more readers than writers, and I have to balance how long it takes for somebody who's interested to read, versus how long it takes for somebody who's not interested to skip.
The new HN "prev" and "next" buttons that were recently added to help make HN posts more accessible to people with screen readers are helpful to everyone else too. (Accessibility helps everybody, not just blind people!)
And as the internet has gotten faster and storage cheaper, while the user interface quality, usability, and smooth flow and interactivity of browsers has stagnated (especially on mobile), the cost of skipping over long post gets lower, while the cost of jumping back and forth between many different links and contexts stays ridiculously and unjustifiably expensive (just ask Ted Nelson). Especially with pay sites, slow sites, and links that have decayed and need to be looked up on archive.com (which itself is quite slow and requires waiting between several clicks).
Another consideration is to make it easier for people to find all the information in one place in the distant future, and for search engines and robots and evil overlord AIs to scan and summarize the entire text.
I think of what I try to do as manually implementing Ted Nelson's, Ivan Sutherland's, Douglas Engelbart's, and Ben Shneiderman's important ideas about "transclusion".
[Oh the irony of this "Transclusion on Transclusion":]
>This article includes a list of general references, but it remains largely unverified because it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations.
>In computer science, transclusion is the inclusion of part or all of an electronic document into one or more other documents by hypertext reference. Transclusion is usually performed when the referencing document is displayed, and is normally automatic and transparent to the end user. The result of transclusion is a single integrated document made of parts assembled dynamically from separate sources, possibly stored on different computers in disparate places.
>Transclusion facilitates modular design: a resource is stored once and distributed for reuse in multiple documents. Updates or corrections to a resource are then reflected in any referencing documents. Ted Nelson coined the term for his 1980 nonlinear book Literary Machines, but the idea of master copy and occurrences was applied 17 years before, in Sketchpad.
I err on the side of transcluding relevant text that I and other people have posted before, instead of just linking to it, because often the links need to be updated or get lost over time, it's clumsy to link into the middle of a page, there's no way to indicate the end of the relevant excerpt, and I can leave out the redundant stuff.
Following links is distracting and costly, so most people aren't going to click on a bunch of inline links, read something, then come back, re-establish their context, and keep on reading from where they left off, since it loses your context and takes a lot of time to flip-flop back and forth (especially on mobile). Today's web browsers make that extremely clumsy and inefficient, and force you wait a long time and lose the flow and context of the text.
So I aspire to simulate Ted Nelson's and other people's ideals with the crude stone knives and bearskins that we're stuck with today:
S1E28 ~ The City On The Edge Of Forever / stone knives and bearskins
>"Captain, I must have some platinum. A small block would be sufficient. Five or six pounds." -Spock
See what I did there? Most people have seen that a million times before, instantly recognize the quote, and don't need to actually click the link to watch the video, but there it is if you haven't, or just like to watch it again. If it's a long video, then I'll use a timecode in the youtube link. And I also take the time to quote and transcribe the most important speech in videos, so you don't have to watch it to get the important points. The most interesting videos deserve their own articles with transcripts and screen snapshots, which I've done on my medium pages.
For example, this is an illustrated transcript of a video of a talk I gave at the 1995 WWDC, including animated gifs showing the interactivity, so you don't have to wade through the whole video to get the important points of the talk, and can quickly scroll through and see the best parts of the video all playing out in parallel, telling most of the story:
1995 Apple World Wide Developers Conference Kaleida Labs ScriptX DreamScape Demo; Apple Worldwide Developers Conference, Don Hopkins, Kaleida Labs
And I made animated gifs of the interesting parts of this video transcript, to show how the pie menus and continuous direct gestural navigation worked:
MediaGraph Demo: MediaGraph Music Navigation with Pie Menus. A prototype developed for Will Wright’s Stupid Fun Club.
Here is another video demo transcript with animated gifs of some related work, another approach to the same general problem of continuous navigation and editing with pie menus and gestures:
iPhone iLoci Memory Palace App, by Don Hopkins @ Mobile Dev Camp
>A talk about iLoci, an iPhone app and server based on the Method of Loci for constructing a Memory Palace, by Don Hopkins, presented at Mobile Dev Camp in Amsterdam, on November 28, 2008.
Dang posted this great link to a video by Ted Nelson explaining the most important ideas of his life's work: document structure, transclusion, and the idea of visible connection in text on screen:
>The original hypertext concept of the 1960s got lost on the way to the Web-- and all current document standards oppose it.
>This is an important fight.
>Ted Nelson: "Here we have a Xanadoc. Right now it's disguised as plain text. But if we want to see connections, here they are. The ones outlined in blue are Xanalinks. They aren't just jumplinks, what other people call hyperlinks. I've called them jumplinks since before the web. You're jumping to you know not where: it's a diving board into the darkness. Whereas Xanalinks visibly connect to other content, with a visible bridge. The other documents open and I can scroll around in them! When I client, I can close them again by clicking. This is of course only one possible interface."
In the 90's, I worked with Ben Shneiderman and Catherine Plaisant at the University of Maryland Human Computer Interaction Lab on a NeWS PostScript-based hypermedia browser and emacs-based authoring tool called "HyperTIES".
HyperTIES had pie menus for navigation and link selection, a multimedia formatted text and graphics browser with multiple coordinated article windows and definition panes, and a multi window Emacs authoring tool with tabs and pie menus for editing the intertwingled databases of documents, graphics, and interactive user interfaces.
Each article had a short required "definition" so you could click on a link to show its definition in a pane and read it before deciding if you wanted to double click to follow the link or not, so you didn't have to lose your context to see where each link leads.
HyperTIES even had embedded interactive graphical PostScript scriptable "applets", like JavaScript+SVG+Canvas, long before Java applets, but implemented using James Gosling's own Emacs and NeWS, instead of his later language Java.
Designing to Facilitate Browsing: A Look Back at the Hyperties Workstation Browser: By Ben Shneiderman, Catherine Plaisant, Rodrigo Botafogo, Don Hopkins, William Weiland. Published in Hypermedia, vol. 3, 2 (1991)101–117.
>Hyperties allows users to traverse textual, graphic or video information resources in an easy way (6,7). The system adopts the ‘embedded menu’ approach (8), in which links are represented by words or parts of images that appear in the document itself. Users merely select highlighted words or objects that interest them, and a brief definition appears at the bottom of the screen. Users may continue reading or ask for the full article (a node in the hypertext network) about the selected topic. An article can be one or several pages long. As users traverse articles Hyperties retains the path history and allows easy and complete reversal. The user’s attention should be focused on the document contents and not on the interface and navigation. Hyperties was designed for use by novices, giving them a sense of confidence and control, but we also sought to make it equally attractive to expert users.
Don Hopkins and pie menus in ~ Spring 1989 on a Sun Workstation, running the NEWS operating system.
>After an 1991 intro by Ben Shneiderman we see the older 1989 demo by Don Hopkins showing many examples of pie menus on a Sun Workstation, running the NEWS operating system.
This is work done at the Human-Computer Interaction Lab at the University of Maryland.
>A pie menu is a menu technique where the items are placed along the circumference of a circle at equal radial distance from the center. Several examples are demonstrated on a Sun running NeWS window system, including the use of pie menus and gestures for window management, the simultaneous entry of 2 arguments (by using angle and distance from the center), scrollable pie menus, precision pie menus, etc. We can see that gestures were possible (with what Don call "mouse ahead" ) so you could make menu selections without even displaying the menu. Don uses an artifact he calls "mousee" so we can see what he is doing but that extra display was only used for the video, i.e. as a user you could make selections with gestures without the menu ever appearing, but the description of those more advanced features was never published.
>Pretty advance for 1989... i.e. life before the Web, when mice were just starting to spread, and you could graduate from the CS department without ever even using one.
>This video was published in the 1991 HCIL video but the demo itself - and recording of the video - dates back to 1989 at least, as pictures appear in the handout of the May 1989 HCIL annual Open House.
>The original Pie Menu paper is Callahan, J., Hopkins, D., Weiser, M., Shneiderman, B., An empirical comparison of pie vs. linear menus, Proc. ACM CHI '88 (Washington, DC) 95-100.
Also Sparks of Innovation in Human-Computer Interaction, Shneiderman, B., Ed., Ablex (June 1993) 79-88.
A later paper mentions some of the more advanced features in an history of the HyperTies system: Shneiderman, B., Plaisant, C., Botafogo, R., Hopkins, D., Weiland, W., Designing to facilitate browsing: a look back at the Hyperties work station browser Hypermedia, vol. 3, 2 (1991)101-117.
>PS: For another fun historic video showing very early embedded graphical links (may be the 1st such link) + revealing all the links/menu items + gestures for page navigation:
>This is the HyperTies-based hypertext version of Schneiderman & Kearsley’s 1989 book “Hypertext Hands-On!” included in the book (on 2 x5.25” disks). Running in DOS on Win XP 32-bit VM.
>A demo by Ben Shneiderman of the widely circulated ACM-published disk “Hypertext on Hypertext”. It contained the full text of the eight papers in the July 1988 Communications of the ACM. The Hyperties software developed at the University of Maryland HCIL was used to author and browse the hypertext data.
Not the question guy. But … Holy smoke. Great article and the in fact jumping in and out is hard. I think the one used by iOS books for check out a word or even a link worked as book is a different software than safari. But safari to another link and back … good analysis and good work !
A productive Forth system, for me, has stack effects commented into each of the words, in Thinking Forth style has a literate style which flows, has plenty of testing words which help you build up mock objects that you can test your words on, and often real tests which tests each word to make sure it does what it's supposed to (e.g. place a file id on the stack, run your word, and then test the file id to make sure it has the contents that you expect.) Developing and testing should happen often and frequently in the REPL; it's a very dynamic process. This style is very different from ALGOL or Lisp style structured programming where it can be annoying to test things (though in Common Lisp you should just be able to jump into a running function and see what's going wrong), but in its stead the compiler gives you a lot of assurance as to what's going on.
DonHopkins on July 7, 2015 | parent | context | favorite | on: Thinking Forth (1984)
Thinking Forth is a FANTASTIC book -- by all means read it to learn the most important universal lessons from Forth, even if you're not going to program in it!
Forth is a "glass box" instead of a "black box", and it's simple enough that you can easily understand EVERYTHING about how it works right down to the most primitive words defined by machine instructions. It like scheme in that it's great for meta programming and creating higher level domain specific languages, but its approach is different enough and much lower level than Scheme that it's worth learning scheme as well as forth, to contrast them for a better perspective.
Another interesting related language is PostScript, which is a lot like Forth in some ways (rpn stack based, separate return and parameter (and dictionary scope) stacks, how the threaded interpreter works, and its extreme simplicity and power) but a lot like scheme in other ways (data is code, polymorphic arrays and dictionaries, typed object references instead of raw untyped pointers, with typed objects bound to names in dictionaries as opposed to typed variables holding values (you can redefine the same name to different types, since the object with its type is associated with the key in a dict, the type is not declared for the variable name itself like C), a safe high level language with bounds checking, garbage collection (in a modern implementation -- old printers tend to use simpler heaps), etc).
PostScript (and scheme) is a lot more of a "black box" than Forth is, since there's a lot of magic stuff going on under the hood that you can't see, to make it seem simple on the surface. And I'd say that on the surface, PostScript is simpler than Forth, because of how it's higher level and you don't have to worry about a lot of details. But Forth is actually extremely simple all the way down!
\ First you should:
FORTH ?KNOW IF
HONK!
ELSE
FORTH LEARN!
THEN
\ Then you can:
BEGIN
FORTH THINK!
AGAIN
“ for example, I do a lot of "converting" in the (let ...) section, then the code works on the results of that. If you "ignore" the (let ...) section, the code is usually reasonably clear. That tends to be difficult in classic Forth as that gets all intermixed.”
Great points. One thing that might help out with correctness is typed forths. But I haven't played with any of them. And I'm not sure if that solves the readability issue when sometimes you have stack manipulations happen very far ahead or behind relevant code.
Probably, if you're doing complex calculations with Forth that have many operands, you should favor left-reductions: reducing the stack should be as eager as possible. Don't pile all the operands into the stack all at once followed by a soup of words to decimate them. The code generation algorithm should be: whenever you write down an operand to be pushed into the stack, then if a reduction is thereby made possible, you should add the word to achieve that reduction.
>For instance, suppose we push 200 operands onto the stack
Then you're already doing something gravely wrong. The data stack should not be used like a data structure. That would be analagous to passing 200 arguments to a function.
PostScript has a special data type called a "mark" just for making dynamically sized arrays. So in PostScript when you see:
[ 1 2 3 ]
What actually happens is that "[" simply pushes a "mark", 1 2 and 3 push numbers (or you could execute any code that pushed any number of any kind of object, including nested arrays and dictionaries), and "]" does the equivalent of "counttomark array astore exch pop" where "counttomark" pushes 3 (the number of items after the mark), "array" consumes 3 and returns an an empty array of 3 elements, and "astore" pops the array, consumes as many stack elements as it is long, puts them into place in the array, and then pushes the modified array back on the stack, then "exch pop" discards the mark and keeps the array.
But you're right, no Forth programmer in their right mind would push 200 items on the stack, and Forth doesn't have a "mark" type (or rather, it doesn't tag objects on the stack with their type like PostScript and Lisp do, so there is no telling the difference between a mark and other types).
> For instance, suppose we push 200 operands onto the stack (o1 o2 o3, ...),
Nobody does that. That's a meaningless example. Most systems don't have stacks that deep anyway.
This is not Forth. It's pointless to reason on things you definitely don't do in Forth because everyone knows it will fail.
It's like arguing against C because you can forcefully cast an integer into a pointer and make a function that expects a pointer to crash the system. Well, duh.
Don't do that, as we often say in the Forth community. Probably the first thing Forth newcomers have to learn is stop doing things that maybe are the right way in some other language they come from, but are definitely wrong in Forth. Don't do that, and you will have one less problem. The best way to solve problems is to avoid them entirely. That's a precious lesson Forth teaches you, sometimes the hard way.
> In a language with syntax
The whole idea of Forth is to stop relying on these things. This is probably the main part of Forth training: stop doing things in such ways that the only viable option is to rely on syntax, type checking and other crutches. Forth programmers have been doing things without them for 40 years.
This is where Forth is a paradigm shift, and it seems that many people don't realize it. They usually claim it is a defect that needs to be fixed, but it is actually a feature.
> This is not Forth. […] It's like arguing against C because you can forcefully cast an integer into a pointer and make a function that expects a pointer to crash the system. Well, duh.
That’s not a great comparison because people absolutely do that in C.
If you read "Thinking in Forth" you'll discover that Forth can be more readable than other languages, if you write in the style that is supported by the language. Of course, if your goal is to use the same algorithm as in, for example, Java, your code will look terrible.
How do you feel about haskell point-free or function oriented programming ? Some times it feels a bit like forth (lots of tiny combinator that can feel obscure).
Just asking, I'm no expert neither in Forth nor Haskell.. just curious about your opinion
This is just personal preference. I find lisp impenetrable and parenthetical soup. But I can read pipelined function calls (in e.g. OCaml or Haskell) easily.
My point is that Forth's general goal is to be a higher-level assembly language for embedded systems.
Common Lisp and Scheme's general goal is to be a high-level programming language for application development (also PL research in Scheme's case).
By general goal I mean what you could ultimately guess the focus is based on the implementation decisions and how the community uses the implementations.
They both do a good job but in very different situations.
At high levels (meaning you've built up some nice Forth words), I disagree. The main difference is that most Lisps and Schemes have a GC, so you're not in charge of allocating/freeing memory like you would in Forth. That doesn't stop Forth words from implicitly referring to a variable that actually contains memory allocation addresses and hiding the actual underlying details of the allocation.
I think Forth certainly starts you out at a level of abstraction only somewhat more rich than an assembler while a Lisp runtime certainly abstracts more away from you, but once a Forth has been built-up, they become similar. Gforth gets a lot of flack in the Forth community for not being minimal, but it's a good example of a feature-rich Forth with support for things like exceptions, filesystems, sockets, and more. The Forth community itself is split into broadly two camps. The minimalist camp, (where Chuck Moore likes to live) which believes in bringing up custom Forths for custom platforms, creates small Forths that try to limit abstraction. The more featureful ANS Forth community tries to focus on a more batteries-included experience and these have more in common with most Common Lisp and Scheme runtimes.
Yeah I can see that but, to make a comparison, just because you can build up high level functions in assembly doesn't make it a comparable language to Python/JavaScript.
I think Forth is unique in that you actually can be both high-level and low-level productive in a single "system", but I can also see the perspective that high-level Forth is more of just a concatenative language than a high level Forth as such. Regardless I think Forth's power is that it can be both a very low level language or a high level runtime.
> My point is that Forth's general goal is to be a higher-level assembly language for embedded systems.
This is strictly speaking not true, even though that is a domain where Forth excels. It isn't rare at all to see Forth as the bootstrap language for bringing up new machines or architectures for instance. And I've seen it used for image processing applications, embedded language inside other products and so on. It's definitely not going to be your language of choice for hacking out the next CRUD app, even though there is no technical reason why you couldn't do that.
Just like Lisp is all about transforming arguments, Forth is all about transforming what's on the stack, and to a certain degree the two are equivalent even though their outward form looks at first glance very different. Forth has an easier and cleaner way to add multitasking to the core language than Lisp does (in my opinion, I'm sure others will disagree).
A more modern implementation of the concepts behind Forth can be found in Factor:
Note that Forth (and other stack languages) are an acquired taste, plenty of people don't like them and a good number of those will simply never get beyond the initial turn-off provided by RPN.
Just to confuse things further, both CL and Forth give you control over parsing (reader macros in CL, direct access to input for immediate words in Forth). So you can make a more structured language out of Forth if you really want to, and you can get rid of parentheses in Lisp if they bother you, with enough effort.
(But, of course, it's less effort to just use a different language that has it out of the box.)
I would beg to differ that that is the “goal” of forth, but I do think that there is truth to the statement in terms of niches best served when used by most programmers.
Writing in forth is a process of creating a domain specific language to solve the problem at hand, so it tends to get quite claustrophobic as you close in on the goal.
Yeah I tried to clarify. I am not an authority on their literal goal. I was trying to describe the result when you look at the systems overall in terms of implementation and use; what they make easy.
Even if so, a compiler can read your Lisp code and find errors that are run-time problems in Forth.
$ cat errors.tl
(defun bad ()
(length (cons 3) x))
$ txr -e '(compile-file "errors")'
errors.tl:2: warning: length: too many arguments: max 1, given 2
errors.tl:2: warning: cons: too few arguments: needs 2, given 1
errors.tl:2: warning: cons: constant expression (cons 3) throws
errors.tl:2: warning: cons: constant expression (cons 3) throws
errors.tl:2: warning: unbound variable x
and this is meeting a fairly low bar; we're not even looking at type checking.
In Lisp I can work in a ahead-of-time compilation model just like C, Pascal or Java where I build the source files to object files, and the editor takes me to places that trigger diagnostics like the above.
What matters in readability is is how easily someone versed in that language can spot the error in bad code, and convince him or herself that good code is good, taking into account and advantage all the available tooling.
Forth is implementable in a tiny amount of space, and can produce fast executions. It can be fast when interpreted, and supports a straightforward compilation model. These are worthwhile qualities. Every computer scientist should know about Forth.
Without a doubt, Forth could compile a file and be able to warn that there is a stack underflow. (E.g. by taking advantage of the stack effect annotations, and check them against the actual effect, based on knowing the semantics of words). It won't be able to pinpoint it very precisely though. Without annotations, it is not easy to determine. A defined word which consumes 17 operands from the stack, and produces 9 in their place can be perfectly correct.
> A defined word which consumes 17 operands from the stack,
It’s been a long while since I last wrote a line of Forth, but issues like these can be countered with some parseable documentation. If the documentation of the word says it consumes 17 operands from the stack and returns 9, the compiler will be able to tell if either code or doc is lying.
I think part of this is the somewhat idiosyncratic indenting style, where closing parens are gathered together instead of their own line like curly brackets in C-style syntax. It ends up being somewhat similar to python, in that if a lisp program is indented "properly" you should be able to ignore the pileup of closing parens that tend to happen at the end of a long expression, and even the open parens, and see the structure for what it is.
As someone who has written a forth, and knows a tiny sliver of what lisp can do, the huge difference is that Forth can't deal with arbitrary data structures in anything close to the ease of Lisp. If you want to ingest a chunk of source code, tokenize, parse and put it all in an abstract syntax tree, you're far better off doing it in lisp.
Forth doesn't really do data structures. Stoical is a forth variant that did, but it died out.
It doesn't hand you a ready-made solution due to its extreme minimalism, but that doesn't mean that you can't have arbitrary data structures (and have them be quite ergonomic).
All it takes is a handful of lines (each) to have words for defining C-like structs, sum types, separate stacks for distinct datatypes, RAII, even generics. You only need to implement these things once, afterwards you can simply reuse that solution.
To expect the language, standard library, or popular frameworks to provide all the abstractions one will ever need is unfortunately a very common attitude these days. In general, but particularly in the case of forth, it is very beneficial to understand that you are allowed to make your own abstractions. Nothing is forcing you to stay on the lowest levels of abstraction, where you only directly utilize tools the language has given you. In fact, such an approach is quite antithetical to writing forth.
There's a very good reason for this "common attitude", and it is network effects with their benefits for productivity. If the standard library includes a useful abstraction, then your code and my code that use that abstraction have an easier time interoperating, and so does the entire rest of the world. That's really handy.
There are a lot of Lisps out there, but even more Forths (like the one you've written), although people like to argue what it means to be a Forth.
One desktop/server Forth that I have enjoyed is 8th and it is a Forth in the since that it is a concatenative language using words and a stack, but it also is like GForth in that it comes with garbage collection, while it also has support for various data structures like arrays/maps/JSON/matrix, file IO, GUI, database, cross compilation...etc. It still takes my brain awhile to use, but my workflow is pretty similar to the one I use with Python.
I remember using some Sun workstations in the 1990's. They Open Firmware boot system used Forth. It was also used for PowerPC Macs but I never had any of those.
I was playing with my old iBook from around 2000, and I needed to dig into the open firmware commands to set up network booting. Back in 2000 I didn’t know or care about Forth but since the pandemic I’ve gone down a few Forth rabbit holes (thanks CollapseOS!) so it was fun to know what was behind the scenes of the boot interface :)
Open Firmware is still used by IBM Power servers, although not as the "actual" bare metal firmware[1]. It's instead the default firmware interface for the standardized "VM", PAPR. A cut-down variant called SLOF is shipped with QEMU for this purpose, either for full emulation, or combined with KVM on Power hardware for accelerated virtualization.
[1] Technically there's nothing preventing bare metal hardware from implementing PAPR, but to my knowledge, no currently produced hardware does. Same direction that Sun went with their servers late into the game, not long before their acquisition; sun4v ran everything inside LDOMs, with the default configuration giving all resources to a single LDOM, IIRC.
Very similar philosophies re building up abstractions, DSLs and DSL-enabling syntax (half of Thinking Forth is essentially proselyzing DSLs), very different philosophies re lowest level of abstraction accessible in the language and compound datatypes. I’d say that neither of those are superficial.
I'm quite familiar with many Lisp dialects, including Emacs Lisp, Common Lisp, and Clojure, and I'm moderately familiar with Scheme too.
I'm not a stranger to other outside the mainstream languages like OCaml and Haskell.
However, every time I've tried Forth I've bounced off it hard. Has anyone here had a similar experience and managed to overcome it? I really want to learn Forth because I believe learning new paradigms makes me a better programmer and it's also fun. So how can I make learning Forth fun?
And on a tangent, it's nice to see posts from the original Wiki. I spent a lot of time on that site a couple decades ago.
I've found that the best way to truly grok Forth conceptually is to implement one yourself - and I don't mean a toy interpreter, but the real thing producing basic threaded code. Then you understand why it is the way it is, and what its strong sides are. Starting with a book that merely describes how to use it like any other PL just leads to questions like "why is this so weird?".
The reason I don't like building a Forth as an introduction to Forth is because it has you stop at the lowest level of abstraction a typical Forth program is written in. Forth is all about building layers of abstraction for yourself. If you're marching toward implementing a system with a set of words others have written about then you aren't really getting the practice in building your own abstractions that help writing Forth applications. It's why a lot of the Forth community laughs that most people come, write a Forth, then leave. A real Forth application can get quite high-level but it's up to you and your collaborators to design that tower of abstraction.
I prefer a different approach myself. Start by writing small scripts, stuff that you've already written in other languages, in a full-featured Forth like Gforth. Once you feel comfortable doing that, start building small applications in Forth. Eventually, write bigger applications in Forth. Only shrink your base Forth vocabulary once you feel comfortable in a full-featured Forth like Gforth.
It depends on what you're looking for. If you think of Forth as just a macro-assembler that's easy to get started with, then you're going to just use Forth as a way to bring-up a system and leave. If you think of Forth as a way to build a custom DSL around essential complexity, then you'll be comfortable taking it as high as you'd like. I know most people, especially the types of folks who experiment with languages, view Forth as the former (especially since a lot of these folks are either interested in something high-level like Lisp or with more static assurance like an ML derivative), but if you want to actually see where Forth shines, then you need to actually write high level code in it.
But, again, if you think Forth as a way to build a custom DSL, why wouldn't you use Lisp instead, which has a lot more to offer in that department to begin with?
OTOH Forth as a universal macro assembler is essentially a unique niche where it not only fits surprisingly well, but it's also obvious as soon as you start using it in that capacity.
The upshot of the discussion seems to be that the definitions of a Lisp and a Forth are so orthogonal that one person correctly say "most lisps have no relations to forth and vice versa" and another can correctly say "look, a lisp that's also a forth".
Bootstrapping a development environment on an exotic and/or very resource-constrained platform.
The neat thing about Forth is that, once you have the kernel running, and can hook some simple I/O to it (e.g serial), you basically have a remote REPL. And this can be made to work on devices where RAM is measured in kilobytes.
Speaking of this, I've always loved this paper [1] about Forth. And it's a real eye opener. It's a real eye opener when you understand that primitives such as assignment and variable references are, indeed, in Forth, actual primitives. Primitives that can be readily changed.
Combine this with a venerable Forth (cross)assembler and it's pretty powerful for all of its simplicity. This is bootstrapping without even a boot, just need a sniglet.
Indeed. In college, I designed a stack based CPU that made it trivially easy to build a Forth compiler for it. It also had registers, because I started with a register based design, but the microcode fleshed out stack based instructions and the registers were mostly for storing data outside the stack.
Chuck is quite fond of indexing registers on his forth machines; they make operations like memory copies much less messy than exclusively keeping state on the parameter stack.
It's a shame it was never built as real hardware (or a relief, as it'd probably have all sort of analog and timing issues), but it was really sweet to program.
I have often thought of writing an interpreter to run a PostScript clone without the graphic primitives but adding some words dealing with files. PostScript does have a lot of fun extras.
Their example of "Forth-like" syntax looks absolutely nothing like any Forth I've used. The "F'" version more closely resembles Factor or Joy, if anything.
Coco Conn and Paul Rother wrote this up about what they did with FORTH at HOMER & Assoc, who made some really classic music videos including Atomic Dog, and hired Charles Moore himself! Here's what Coco Conn posted about it, and some
discussion and links about it that I'm including with her permission:
>First shown at the 1989 Siggraph Electronic Theater to a rave response, this 3 minute humourous film went on to win several top computer graphic awards that same year including Niccograph of Japan.
>Coco: This was a show favorite at the SIGGRAPH film show that year. The year before the conference committee decided that showing demos wasn't the way to go anymore. Peter wrote Flying Logos as a way to sneak our demo reel into the show by turning it into a story. It worked and we made it into the film show.
>Don: I truly believe that in some other alternate dimension, there is a Flying Logo Heaven where the souls of dead flying logos go, where they dramatically promenade and swoop and spin around each other in pomp and pageantry to bombastic theme music. It would make a great screen saver, at least! Somewhere the Sun Logo and the SGI Logo are still dancing together.
----
Peter Conn and I [Coco Conn] had a company called HOMER & Assoc. which was located at the Sunset Gower Studios from 1977 until we closed shop in 1997. We made music videos, commercials & computer graphics/special effects for feature films. One cool note, we worked with Paul Verhoven on both RoboCop in 1986 and the x-ray scene for Total Recall in '89.
HOMER was actually a real time visual mixing console that our in-house engineer spent 1978 - 1981 designing and building, from scratch. The name HOMER stood for "Hybrid Optical Montage Electronically Reproduced." I helped as well, soldering the LEDs on the console and running cables. Peter built his own optical printer and three years into the build we also bought an early computer paint system. Our engineer finished building the console and promptly decided to move to England. We hadn’t used it because we still hadn’t found the right software to run the system. Luckily that’s when Paul Rother joined the company.
The joy stick on our console would bump you to the next line of code (being a command or sequence of events: fade, cut, dissolve, etc.) The console had touch sensitive fader pads. There were no dials. I think they were made by Allison? Each channel (which controlled either a slide projector or a film projector) was touch sensitive. After recording a sequence we could then tweek the current version using additional effects the channels offered such as momentary, additive, on/off, etc. For instance if you wanted to crossfade two images, you could either program it or perform it. Of course everything you did was recorded and would play back on the next round. You literally performed a sequence of visual effects with your hands. Peter would do countless passes until everything was perfect. This performance would then be played back to IP film on the optical printer. Each slide tray or film real would be individually run, one by one, to IP film. Sometimes there would be 10-15 or more passes to get all the elements transferred. Once that was done we would then convert the IP film to video and do additional video editing and effects. A totally nuts analogue system. But it worked.
---------------
HOMER Explained by Paul Rother, in-house programmer, (1982):
The photo is Paul sitting in front of the Optical Printer 7-bit Paint system, Homer and Associates, circa 1982.
Homer and Associates was really one of a kind kinda of company. Founded by Peter Conn, originally I got hired to program Homer II, a visual realtime mixing console. Homer I is another whole story, but before my time.
Homer II consisted of 16 slide projectors, 4 movie projectors, a 4 track tape recorder, 24 visual channels (each with its own Z80) touch sensitive sliders, a master Z80 S100 bus system and featuring "the joy stick bumper " control, which looked liked the gear shift right out of a 1964 mustang convertible.
The idea was that you would program a visual sequence, then play the sequence in sync with the sound track on the joystick, including cascades, bumps, cuts, etc. The whole thing would be recorded, and if you wanted to, like an audio mixer, go back and do over dubs, making corrections. Then once you had the perfect "hero" recording, you take the 8" floppy disc with the hero recording and the trays of slides to the optical printer, and record it to IP motion picture film, making multiple passes, one tray at a time. Now that I think about it, it was a crazy idea. We actually got the whole thing to work. And it worked great!
Forth & Charles Moore
We hired Forth, Inc. and got Charles Moore, the inventor of FORTH to program the console host computer. I learned FORTH and worked with Charles. I programmed the 2K byte EPROM in each visual channel. On the Master Z80 system we ran PolyForth a multi tasking system in 32K bytes. We had an extra 16K RAM for buffers and things. If I remember right, the system ran four tasks, but that was 20 years ago, my memory may be hazy.
Anyway, I learn not only FORTH from Charles Moore, but also how to factor code in to small reusable routines, WORDs they're called in FORTH. I learned Object Oriented Programming without knowing it. Also a lot of use of vectors. Its a cool language. Charles Moore was a great inspiration to me, and really taught me a great deal that they never taught me in computer programming school.
CAT-700
After we got the basic Homer II working and were able to record on the optical printer, Peter had another idea. He wanted to be able to see the movement of the optical printer, and see a prior frame compared to the current frame. We already had a video assist on the Fries Mitchell 35mm. What we needed was a Frame Buffer. We heard of S100 video board called the CAT-100, which was 1-bit frame buffer, good enough for what we needed. Somehow we never found a 1-bit version, but we found 7-bit version in the recycler!
We flew to Reno, rented a car and drove to a log cabin up in the hills of Truckie California. We got a demo of the thing. The guys were super secret and didn't want us to see the controlling program. It worked, so we bought it, and then flew onto Palo-Alto and met the French guy who designed it. They checked it out and it was OK. This was the days before computer designed boards, and all the traces on the board were curvy, kinda like a Van Gogh painting. We learned that it was 7-bit (CAT-700) because it would have been an 8-bit, but they could not get the 8th bit to work. We spent the night in Palo Alto with a Stanford friend of Peters working on a crazy secret Apple project, the Lisa.
32KByte Paint System
So I got the CAT-700 frame buffer to work, programmed in FORTH. So in that 32K we had an optical printer control system, and a paint system, all in one. (Also the OS, compiler, debugger, etc.) We later hooked up a Summigraphic Bitpad (before the Watcom tablet) and were able to draw on top of digitized frames. It got to the point where we needed TWO optical printers, one to digitize from film, and the other to record to film. Rube Goldberg is not strong enough descriptive to describe the system, with the filter wheels and all on stepper motors, it made music. The first use of the system was effects for Steve Miller Music Video, Abracadabra. I also remember using it on the George Clinton Video, Atomic Dog.
This photo was taken right after we got the system to work. I had hooked up an analog slider box, which controlled things like color. There were 4 color maps we could switch between instantly We did a lot of work in planes, using 2 planes for the original image to be rotoscoped, and the other 5 planes to draw onto. This photo was taken for an article in Millimeter Magazine. The photo ended up being a two page color spread, and I think Peter was pissed, cause I got premier exposure.
TTL logic
At Homer and Assoc. I also learned TTL logic and designed a number of computer boards for the S100 bus. One that controlled stepper motors with a timer chip (Motorola 6840). Another to control the Slide Projectors also using the same Motorola timer chip to control the lamp triacs. My favorite thing, about the system, was the use of the cassette storage interface as a cheap timecode reader/writer.
The entire Python default windowing system relies on TCL, and companies like AMD use it for a lot of their internal test frameworks because it's easy to implement and well supported. So for embedded, which is people's argument for fourth, TCL is way more useful as evidenced by it winning the industry.
Extremely widespread embedded support and cross platform. Forth seems dead to me, whereas TCL is huge and has good docs, a good standard library with batteries included, and it's easy to implement on embedded hardware because of how well documented it is. Also the inherent linkage to Python keeps it relevant.
I find forth basically impossible to read even when, at best, authors leave a comment on basically every 1-2 words. Writing it is pretty hard too.
In contrast it's about as easy to be productive in a lisp (CL or Scheme) as it is in Python or JavaScript (give or take useful 3rd party libraries).
Even if they're from the same time period and are interesting for being not very Algol-like, I don't think it makes a lot of sense to discuss Forth and Lisp in the same conversation. They were both improvements on very different things.