Tcl wasn't my first language, but it's near and dear to my heart. It's a little oddball for sure.
I can't decide whether it's Python trying to be Bash, or Bash trying to be Python. But it's surprisingly expressive.
Everything is strange about the syntax - from "everything is a string" to "expr" to the weird variable-assignment syntax.
It's also really tiny, and has a great async event-loop framework. It has built-in support for sockets and non-blocking I/O. It has a GUI toolkit that's pretty useful for making quick-and-dirty tools. It's incredibly easy to use interactively with a REPL. And it's still the scripting language of choice for the FPGA industry, which is where I got my start as a software engineer.
The Tcl community is small and stable. And the language just keeps soldiering onwards, in a gradual way. It feels like it's been stable and fully-realized for the last 20 years. No breaking API changes, no syntax changes. It just keeps on keeping on, providing automation to tools that need it. Tcl has outlived a bunch of newer languages, and I hope it sticks around for another 20 years.
I have a strange test for languages - I tried some of them being drunk (a litre of some nice tripel). This test tests how effective can you be with diminished higher-order thinking, or, in other words, how much of your mental ability each tested language takes from you.
Tcl has immutable proc arguments - if you pass somewhere a value you can be sure the value at caller's site stays the same. It happened that this means a lot and one can (semi)successfully program in Tcl while being drunk.
Every language where arguments' values modification is visible to the caller is much worse in that regard, these are C, C++, C#, Java, Python and Perl (and many more). Aother bad feature is a null pointer - Per is better here.
Tcl does not have modifiable argument values except you are really want them (upvar and uplevel). It also has multivalue assignment in the form of "foreach {resultpart1 resultpart2} [procedure arg1 arg2] break" (or you can invent your own mutivalue assignment procedure). This makes building composable libraries quite easy.
The immutability and the lack of null are a big part of why Tcl is my favorite scripting language. I learned other languages at a similar level of abstraction before Tcl, but now when I program in Python or JavaScript their mutable collections feel like an obviously wrong default.
[upvar 1] and [uplevel 1], which, respectively, link a variable in your caller's stack frame to one in yours and run a command in the caller's stack frame, usually compose nicely. They are Tcl's replacement for macros and the lack of a compile time/run time distinction makes them, in a way, easier to reason about. However, [upvar N] and [uplevel N] aren't limited to N = 1, your immediate caller's stack frame. You can reach up an arbitrary number of stack frames. It allows you to design really complicated control structures. [upvar] links also nest. You can fall into a pattern of carrying around some sort of a context or configuration data with nested [upvar]s like a poor man's Reader monad. Both are difficult to debug when they go wrong and should be implemented with care and only when necessary.
Since version 8.5 Tcl has a built-in command for assigning list elements to variables:
Tcl is _famous_ for allowing to program in it while drunk (and get some good results). This is why it is somewhat popular in the academia / scientific engineering circles. :)
> Python trying to be Bash, or Bash trying to be Python
The fact that "everything is a string" is cute and odd but the end result is really closer to shell scripting than a real object-oriented language like Python. You routinely have to deal with string/list quoting and constructing complex data structures is tricky if they have to be represented with just strings and hashes.
Sure: it's flexible and you can reimplement all the features of a high-level language but I'd rather use a language which has all the features already.
Tcl has native support for lists and dicts, and provides full-featured functions for working with both. You're absolutely right that the feel is "more than Bash, less than Python." But it's not accurate to say that you need to do a bunch of weird string parsing in order to implement a high-level datatype.
And yes - the list quoting rules are weird! No doubt about that.
I have an old engineering program that uses tcl as its scripting language... stuck on a version that predates dictionaries.
Without dictionaries complex data structures are quite difficult, and people will abuse the heck out of the language to create them. I was just looking at an example where some poor engineer was inventing a "data structure" by using string manipulation to create & modify variable names. Some of the variable names he derived were also defined as globals, so there was that too.
Prior to dictionaries, tcl only had arrays, which are the least string-like things in the language.
> And yes - the list quoting rules are weird! No doubt about that.
Lists might be "native" in the sense that the interpreter knows how to deal with them but in the data model there is no distinction between a "string" and a "list of strings". A list is just a space-separate string with quoting rules and the difference is entirely in manipulation.
It's easy to make quoting mistakes so a program might start breaking when arguments contains spaces: this almost never happens in languages other than shell.
> A list is just a space-separate string with quoting rules and the difference is entirely in manipulation.
This has not been true since the release of Tcl 8.0 that added the Tcl_Obj subsystem.
Lists in modern Tcl (8.0+, with 8.0 released in 1997) are proper O(1) indexed arrays. Yes, you can still request from Tcl the 'string' representation of the O(1) indexed array, and the result you get is the old (pre 8.0) "space-separate [sic] string with quoting rules" variant that will parse back into the O(1) indexed array later if you want.
But Tcl lists have not been /only/ those strings for a very long time.
Dicts are implemented using hash tables and thus slow on pass to procedure that modify them. I had to switch to Python and then C when I encountered quadratic runtime using dicts.
I hope they will use something like hash-array-mapped trees someday. It will make things much faster.
Thats not quite true since the early 2000s, when dictionary structures came around. Tcl also has strong built in object oriented support since 8.6 (2012) so if you like that sort of thing its there.
If you are dealing with quoting issues that's probably due to you using an older version like 8.3 or 8.4. Give a newer version a shot!
Tcl/Tk was a tremendous rapid GUI prototyping tool. I remember cobbling together a Verilog schematic viewer in a couple of weeks using that combination, for the Series A funding round of the startup I was working at in the early 2000s.
This was my reason for foraying into Tcl too. Till then I had used Java's AWT and Swing; but prototyping with Tcl/Tk was way faster and intuitive. Mid-2000s. I ended up liking the language - idiosyncratic as it was. Partly the reason I am curious about and planning to read Ousterhout's (creator of Tcl) book "A Philosophy of software Design" soon.
The event handler based vector graphics in Tk were unmatched until HTML 5 and the canvas. And then there was the easy and fantastically well documented C extensibility...
Tagging system of Tk's canvas is unmatched in HTML. Pity they (HTML designers) never learn.
Basically, in Tk, you can add a tag to an element of your drawing in canvas and assign mouse (or keyboard) event handlers to change colors or other attributes on entry and exit of mouse cursor.
I made a OO/relational database schema visualization in Tk, it was breese to navigate - once you hover over a relation, you can see classes highlighted; follow highlighted relation to get to them. And once you highlight class of objects you can see to which relations it belongs.
It tooks two foreach loops to get that behavior. As far as I know, HTML5 canvas does not support that out of the box.
I think it still is. A very easy to use, quite well working cross-platform solution. Since 8.6, the UI has picked up a much more nativel look and feel.
You have no idea how much I wish i could interact with a simulator using something other than tcl, preferably python.
I don't have anything against tcl per se, but I can't think nearly effectively about anything because tcl doesn't really live anywhere else that I work. I can appreciate that it has the audacity to thrive in the tools I have to use, but I always sort of pine for what could be.
Tcl is good for interacting with a simulator (and other tools) for the same reasons Bash is good for interactive shells, and I think Tcl is the best tool for those jobs.
so like one instance I had is making a big input vector for running my testbench against. JSON, CSV, work with it in a dict or list, anything that I sort of already knew in python was better than what I cobbled together.
It's nothing against TCL, and it no doubt has the power to do it, it's just up I do FPGA stuff, python is my swiss army bludgeoning device. It's just like EDA tooling is like, the #1 place where I wish python lived, but it's TCL's stronghold.
> And yet I still can't get my head around upvars.
I haven't used Tcl for maybe 15 years, but if I recall correctly upvar was basically an alternative to lack of pointers.
What it was doing was giving you access to a variable from specific stack frame.
You could say I want to access variable from the function that called me.
upvar and it's interesting syntax allowed to create new "statements" for example in earlier versions of tcl (because looks like it was added later) you could implement try-catch "statement" and it would behave as if it was part of the language.
> upvar and it's interesting syntax allowed to create new "statements" for example in earlier versions of tcl (because looks like it was added later) you could implement try-catch "statement" and it would behave as if it was part of the language.
This is one of the things I miss most when using other languages. Outside of actual lisps (of which I consider Tcl to be one), there's very few languages that allow you to build your own control structures. Ruby comes close, but doesn't really have the same flexibility.
Not really, I've come from perl. My half built web project is still sitting stable as ever since not touching it in many months, looking at it; it's like entering a overgrown temple. Knowing if you make an alteration the whole temple can collapse.
TCL just seemed to fit and discovering naviserver could do "register domain/home execute procedure return html" without having to create a mess of routes, models and all the rest just kind of made me happy.
> It's also really tiny, and has a great async event-loop framework. It has built-in support for sockets and non-blocking I/O. It has a GUI toolkit that's pretty useful for making quick-and-dirty tools.
There are implicit "OR" statements between those sentences.
Example: suppose a newbie wants to use tk do make a quick-and-dirty GUI and leverage tcl's async event loop to make it responsive while processing a bunch of files. It's my experience that while this can be done it is less than easy.
This can be tricky, because programming based on events tends to push you into writing everything as event-handlers, but they get awkward for long-running stateful computations. However Tcl has had coroutines for about 10 years now, and those allow you to bypass this problem completely.
I gave short presentation about applying this approach to scripts combining a responsive GUI with managing multiple parallel Expect sessions at www.eurotcl.eu/program.html#Macleod .
I don't recall that being the case. Has something changed in the past years that prevents you from using the same event loop for both gui and async behavior?
And if it's an ssh session you want to remote control, paramiko-expect is great, in particular the take_control method that lets your remote session be partly automated and partly controlled by the user running the outer program:
I like pexpect but it's next to useless on windows. If you are mostly doing ssh/ftp then paramiko is a useful (if heavy) pexpect replacement and works across platforms
Years back I worked on an automated test framework for small/mid-business scale cisco-clone routers that used Expect to drive the telnet CLI interface and put the routers through their paces.
I remember something a more senior engineer at the company told me: Expect is the best tool for when you need to get a job done the wrong way. (Using Expect was his idea, to be clear.)
Curious... your colleague said Expect was "the best wrong way", what would have been the "correct" way in your scenario? How would something like that best be done now? Ansible?
And modern software that does the same. Cisco AnyConnect VPN thing, for example. You can feed it a command file, but that doesn’t give you any error handling or logic to handle dynamic responses. Expect to the rescue!
> The SQLite library is designed to be very easy to use from a Tcl or Tcl/Tk script. SQLite began as a Tcl extension and the primary test suite for SQLite is written in TCL. SQLite can be used with any programming language, but its connections to TCL run deep.
Given both of these, it kind of feels like something about the Tcl philosophy leads people to build (or draws in the sort of person who builds) simple/elegant data storage layers. Makes me more curious about it.
Ousterhout's book on Tcl has a chapter one that is essentially "How to do small reusable things in unix and win" - it's absolutely brilliant and explains how to think in unix and why composable entities with simple interfaces have done so well. SQLite and TCL are the best of unix philosophy made concrete. I just read in this thread that he expanded it to an entire book - I'll be reading that as soon as I finish binging The Family Man on Prime Video. Extending TCL with a new C command is exactly as hard as a CS 101 C program (which at the time was the lower limit on easy for CS folk).
This paper by John Ousterhout (creator of Tcl) is a good read if you think Tcl is a weird language plagued by bad design decisions. If that describes you, it's probably the case that you don't understand the problem Tcl was designed to solve.
Tcl is an alien technology I wish I embraced when I was first exposed to it. I didn't quite get it, I was confused, so I gravitated towards Perl because it reminded me of C somewhat. I wasn't aware of my cognitive blind spots. In many ways Tcl is more powerful than most implementations of Lisp. Like a Lisp crossed with a Forth with the ability to turn spheres inside out with no tearing during execution. Magic.
You probably will have to run the Listener and use its Browser to go through the documentation to see how awesome it is. That, or just look around here:
The first link might give the wrong impression about Factor not being low-level, but it definitely can be if you want to; after all, its optimizing compiler is written in Factor itself!
One thing those three languages have in common is that they are almost syntax-less. Each, in its own particular way, allows user defined code to process tokens of the program source before it gets to the core interpreter, allowing users to implement custom control structures and DSLs.
Although I love Tcl and find it very powerful, it is not as powerful and flexible as Lisp languages. It doesn't have nested procs, closures, lambdas (although one can [eval] or use [apply]) and other functional stuff.
I think the article doesn't make enough mention of the fact that internally, most things are not actually strings for much of their lifetime. They must be able to be represented that way, but that's different than everything being a string all the time. It does create a few problems, but they're a bit more subtle than "OMG, the integers are strings all the time!"
This is an old wiki entry that dates from the pre-8.0 days of Tcl (The history is truncated). For the past 20 years everything has been representable as a string.
I recall when we were porting our site to tcl 8 that one of our devs was giggling uncontrollably when he announced in our morning meeting that he'd created a test environment to shake out any bugs in the upcoming tcl 8.
Right. I think the one great feature it shapes with Lisp is, that everything has a textual representation. That means, all types of values can be presented as text and can be recreated from a textual representation. For Lisp, it is S-Expressions, which can be printed and read, for TCL it is the syntax which can be printed as strings and interpreting strings as data.
>I think the one great feature it shapes with Lisp is, that everything has a textual representation.
Is this true even for procedures? I was recently playing around with Scheme and was disappointed that once a procedure is defined, it got internalized and didn't have a textual representation other than #<proc> or the like.
In TCL, procedure signatures and bodies are stored within the runtime. You could essentially dump the entire state to a TCL file.
For me Tcl's killer app has been Expect: https://core.tcl-lang.org/expect/index there is nothing quite like it if you have to manage 1000's of devices and cli's are the only thing you have to do it!
My first job was in Tcl. I grew to really like the language, though when I had to leave the company I was at a pretty strong disadvantage due to how uncommon it is.
Everything is a string is a great way to reduce the mental load about typing.
One of my earlier jobs was porting Travelocity.com from TCL to Java - the TCL programmers loved the old site and absolutely hated the "new" one. I don't think I've ever managed to make more enemies in my professional life than I did in those couple of years.
Tcl to Java is a huge change. Totally different syntax / syntactic style. I can imagine the annoyance of the Tcl programmers having to deal with the inherant verbose-ness of java.
Yeah, and this was in the relatively early days of Java, too, so the front-end was all JSP. Oh, did I mention they originally gave us a three-month deadline to port the ENTIRE site?
> Everything is a string is a great way to reduce the mental load about typing.
I feel almost the exact opposite way...
Everything is a string means I have to "know" whether any given function takes {a single element, a list of elements, a glob, a list of globs, ???}.
When I inevitably mess it up, I don't get any help at all from the language. In the best case, I'll hit a run-time error and figure it out. In the worst case, my program will just do some random thing that isn't what I wanted, and I'll get to debug it.
The Tcl/Tk documentation is very good, so I read the documentation for those details, eg for lrange,
lrange list first last
List must be a valid Tcl list. This command will return a new list consisting of elements first through last, inclusive ... (not put it all here because it goes on a bit :)
Sure. Sadly my company's code base, and the vendor's tool that it extends, are not so well documented. A statically typed language gives just you so much more sanity checking -- sometimes even without having to compile anything, what with these fancy IDEs.
The SQLite author is a big Tcl user. I believe the codebase is primarily Tcl when you include the test suite, though the core implementation is obviously a C library.
D Richard Hipp is his name and he has some nice talks online. He uses his own text editor and chat app he wrote in Tcl to maintain the project and communicate with other developers.
From what I understand, D Richard Hipp's appreciation of Tcl is the reason SQLite has dynamic typing. (I believe I heard this in one of his talks that's on youtube, which are excellent.)
That makes sense. I heard him say that the pain points of dynamic typing are mostly just people using it for web apps that eventually have to migrate to Postgres and the like, causing some headaches because client server DB’s are much more rigid, causing errors.
I agree that his talks are great! He’s such an enthusiastic guy.
I hope SQLite will adopt default foreign keys and WAL sometime soon, but I understand that he needs to keep it as backwards compatible as possible.
Don't forget the amazing AOLserver, which was a very elegant web applications server with Tcl as the native template language. This was a great environment.
Hey! I wrote flap in TCL module for it! Loved using AOLserver but writing the extension taught me to fear threads - now loving Golang :) but I miss AOL, AOLserver, SAPI and TCL in the evenings.
I have about 5 years of heavy experience with TCL, split in half by 15 years away from it. I got a lot done with it, and it is an interesting language, but I have never, ever said, "I wish I was using TCL" in any situation. It is a unique journey down an alternate path for computer languages, that really ends up nowhere compelling, but the path itself is not arduous, although the bugs can be.
I had 1 year of full-time experience with TCL, and I can basically summarize it in the same way as you: got a lot done with it, wasn't horrible, but don't miss it.
In retrospect, I am kind of happy that somebody actually went and built TCL. "Everything is a string" is not really a sound PL paradigm, but someone needed to get stuff done, and it worked quite fine for that. It's super simple and super powerful, and beginners can be productive with it really quickly.
Similar, I was forced to write scripts in it when I was at a more silicon/hardware oriented company. It's an interesting language but I much preferred moving on to ruby/python for my V&V activities.
> This simple and unified perspective on type is fundamental to the design of Tcl, and puts it in an exclusive category programming languages that do not have at least two data types that can be distinguished. The other languages in this category are forth, BCPL, and assembly language.
It's nice to guarantee that every value has a string representation. The not-so-nice part is that you have no static types and therefore no easy way to guard against mixing up safe and unsafe strings, so security reviewers will have a hard time. This is a design from before when the dangers of handling untrusted input as bare strings and doing escape handling all over the codebase were understood.
The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. -- Alan Perlis
I think Tcl strings aren't exactly what Perlis was talking about. First, the language knows how they're used, so there's a JIT (or whatever) behind the scenes eliminating the literal deserialize-process-serialize pattern you see in shell scripts. Second, a lot of Tcl data types (hash tables, sockets, etc) can be represented by strings but the string is just a simple reference to a rich internal structure. It's almost more like a pointer than a string in other languages.
I know this item is old but surely no shell script should be limited by string copying of the language? Maybe string shuffling thru pipes, but that's the reason the CPU briefly spikes.
I had a bit of trouble knowing which "escape level" I'm in while debugging. You have strings that wrap strings that wrap strings, etc.
Maybe with more experience I'd learn to manage string nesting levels better, but there does seem to be a non-trivial learning curve, at least for us mortals.
Everything being a string at a language level is similar to the paradigm as 'everything is an object' in java. Outside of select use-cases (scripts/text proc), the language generally fights against both conciseness and performance.
I worked with UCLA where a few teams needed to use Tcl to maintain existing web servers, and virtually every developer highly complained about using it (aside from one or two career Tcl-ers). To be honest, I don't personally have much experience with Tcl, but from what I've seen/heard, I wouldn't ever think of building web servers upon it.
One of my first jobs after college involved creating websites using Vignette StoryServer. At the time Tcl was the only language you could use with it. I was happy when a new version was released that supported Java and ASP. Tcl looked like voodoo magic to me but I'm sure for those who really got it, they were pretty productive with it.
You've brought back some painful memories for me. I'm permanently allergic to the words "lindex" and "upvar", and cringe whenever I see a "," character in a URL...
Vignette's only redeeming qualities were that: (1) the page templates permitted the execution of arbitrary Tcl code, not just the small subset that was documented by Vignette; and (2) much of the Vignette functionality was itself actually implemented in Tcl.
This led to the glorious discovery that one could simply bypass the entire crazy Vignette system and use it like a very expensive (and somewhat slower) AOLserver. I sometimes feel sorry for the poor souls who inherited my terrifying and entirely undocumented creation, but that's what happens when management forces developers to use the entirely wrong tool for the task at hand. ("We spent $100k USD on this software so we're damn well going to use it!")
I think it is worth mentioning that I learned TCL through the Pure Data audio language, which was designed with the idea of continuously supporting at least 20 years of hardware IIRC.
I have never used Tcl but this headline truly intrigued me. It sounds like the antithesis of JavaScript's "everything is an object". An object is very abstract whereas a string is very ... literal? I know many generic operations I can already do with a string but then inspecting its contents (which would be a generic operation) may reveal other capabilities, restrictions, what have you. Hmm ...
For most of the past 38 years I have earned my living as a computer programmer/developer using the Mumps programming language. It, too, views most everything as a string (or an array), and sometimes that string can also function as a number.
That's really weird. I didn't know this was a thing but I had a dream a couple days ago that I was using a language in which everything was a string (like Lisp but with strings instead of lists). Guess I'll have to give it a whirl.
It matters even less for Tcl, as it's supposed to tie pieces written in more performant languages together. See "Ousterhout's dichotomy"[1].
Sadly that also moved it closer to something like Rexx (or Lua these days), and thus no proper module/package community ever arose in time.
And Perl has PDL.
Not aware of anything for Tcl beyond math::statistics, but I think it waned in popularity way before the ML bubble started inflating.
But in my experience, the strategy for Tcl is the inverse of Python when it comes to domming/subbing with your "partner language". You _extend_ python, you _embed_ Tcl.
At least in the areas where Tcl still survives. It lost long ago against the P-languages, but still faces Lua rather bravely in the enterprise/engineering sector.
TCL had an amazing architecture for someone who was working in C. In C, you'd (1) create an instance of a TCL interpreter, (2) register your C functions with the TCL interpreter, and then (3) load scripts into the TCL interpreter. You'd do all your fast stuff in your C code, and just use the TCL for stitching together the high-level logic.
Especially the commands were invoked command_name(argv, argc) and the thing was just like a CS 101 C program. Dead simple, and as they mentioned just used to orchestrate the fast stuff. You shouldn't make a command block a long time, but you could grab a handle to the output destination and set it up to stream output there from the C.
I believe they are referring to the Tk GUI toolkit, which is intended for use from Tcl and really nice to work with. It is full featured and has plenty of widgets and layout controllers. You can find out all about Tk here: https://tkdocs.com/
It is really fast and easy to create GUIs using Tcl/Tk, e.g. here is a Tcl script to create a paint-like program in less than 50 lines: https://pastebin.com/6qibnQcJ
The main downside is that the look and feel is not native, and a bit dated. There are some efforts to update the look, but it's not as slick as Gtk etc.
I think it's cheating a bit to compare JIT vs runtime script evaluation. I'd be more interested to see LuaJIT performance measured against something more comparable, like JS running on V8.
About a decade ago now, I was writing a group chat system of the Slack/Discord variety, and I was pondering how I'd like to support markup. I wanted to do something more powerful than just the regular "bold/emphasis/strikeout/links/etc." set, as the users of this group-chat were the type who would enjoy adding animations and other "effects" to their text, especially on top of their custom emoticons/stickers. (Don't fret, this wasn't team-productivity software ;)
What I ended up doing was extending the backend with support for parsing messages using a "command language"—essentially Tcl. The "everything is a string" feature of Tcl-like languages was essential, because I wanted these "commands" to be arbitrarily composable, and both the input and output were required to be strings (i.e. HTML), so commands couldn't pass data-structures to one-another; they had to accept strings, and emit strings.
The user was not allowed to enter arbitrary HTML in their message (this was filtered out as a pre-filtering step), but any Tcl command was allowed to take HTML-strings as input and emit HTML-strings as output, and the string that was eventually generated, HTML and all, was treated as the user's actual message. Many of the implemented commands would parse the input HTML-string into an AST, tweak it somehow, and then re-emit it as a string.
An example of such a command would be something like:
And then [spoiler the protagonist died [rotate 5 [big :ohno:]]]
...where 'rotate' and 'big' layer together CSS effects (combining CSS classes with a bit of custom element style to parameterize it, with each rewriting the HTML AST structure of the previous to ensure the CSS effects "stack" correctly); and 'spoiler' is just a CSS class with pseudo-selector behaviors.
Another example would be:
[type oh nooo[sfx fall]oooooooo]
...which would use a CSS class+JS trigger to display the message one character at a time the first time it became viewport-visible... and embed a hidden <audio> element at the top of the message body, that would get triggered as the typing progressed to it. Sort of like the dialogue event-scripting in a video game.
And, of course, just like an IRC bot, you could rely on the backend being Turing complete, in ways like:
# generate random numbers
[roll 1d6]
# set a countdown timer on a user,
# resetting it whenever the user talks
[deadmanswitch 5mins I'm slacking off again!]
# get the backend to (use a job queue to)
# fetch a resource, plop it into its own download folder,
# and then post a link to the retrieved copy
[dl http://example.com]
# same, but also create a BitTorrent torrent for
# the file, register it with a BT tracker on
# the backend, and then emit the URL of the torrent file
[torrent [dl http://example.com]]
---
At this point, I don't really think there's much room for a new group chat system; but this one feature was really cool to me, and I always wish I had seen it implemented in some bigger product.
I'm pretty sure this hasn't been true for 20 years. Tcl 8.0 introduced a form of byte compilation. [1] While the comments might slow down the compilation a bit, I doubt they have any impact at runtime these days.
This has not been true since the introduction of the bytecode compiler in Tcl 8.4 (well over ten years ago now).
Tcl 8.4+ will byte code compile the proc's, and part of that is removing the comments from the code execution path. So comments only get 'seen' once (at byte code compilation time, upon first use of a proc) and then have no effect on execution performance from that point forward.
This business falls into the "well that's handy" followed by the "who in the world thought honoring continuation characters in comments was a good idea?!"
#! /bin/sh
# the next line restarts using tclsh \
exec tclsh "$0" ${1+"$@"}
And the nice thing about TCL as a language is there's just like 3 or 4 rules to remember and then you can embed the parser in your head. # is a command that discards all its arguments. So it's in the normal flow of syntax, so line continuation works. It's odd to see but easy to reason about after you are used to it.
IIRC this was also the case for node/V8/I don't recall:
Whether a function was inlineable was decided based on its size - and that was determined based on size-as-text instead of size-as-AST.
This lead to the situation where adding too much comment text (I believe it was 512 bytes total for the function) led to the inliner refusing to inline the code, and thus performance regressions.
They fixed it later, but in the beginning, everything was a C string. Meaning you couldn't output a NULL char. Pretty frustrating when I discovered this while trying to write a length encoded protocol :)
Yet UNIX pipes are strings and amazingly popular. Its amazing how we don't see a movement to get typed streams so we can give up the reparse culture of UNIX. I guess PowerShell does typing.
The problem with typed streams is that couples the implementations together on both ends and tends to de-genericise them. You get a "grep" that would only work on strings when actually you'd be quite happy grepping numbers 99% of the time. Also in order to serialise them you'd need fully typed files. Notice that you can't serialise a Powershell stream.
The world tried "everything is XML" and more recently "everything is JSON" as a solution to this. The latter is almost there.
What CLI userlands (POSIX, Powershell, Plan9) are missing, IMHO, is a self-describing serialization "stream container" format, like Avro (https://avro.apache.org), the interchange format used in Hadoop, Kafka, and several other systems of the "gluing IO-streaming components of different languages together into an ETL pipeline" variety. (Which is, of course, exactly what you're doing when you write a Unix pipeline, just in-the-small.)
Where in self-describing data formats like JSON or XML (or even the more efficient encodings like ASN.1), every term is "described in place", taking up a lot of encoding overhead and bandwidth; in a self-describing stream container format, each encoded stream first encodes a schema (or the ID of one) for what it's about to transmit; and then transmits the terms encoded using the schema.
Because schemas are referred to by embedding them in the document in a normalized form, each stream received by a client can be handled by a hybrid approach between building up just-in-time dynamic parsers from the schema, if it's not known; or recognizing known schemas (by e.g. hashing the representation of the schema, which works since it's in a normal form) and using a baked-in decoder for that specific schema if available.
This would work pretty well as an enhancement to standard POSIX CLI IO-streaming tools; they'd be able to use specific logic to optimize the parsing of a few schema-encoded types they "expect", while also faithfully (but less efficiently) handling data of any type by falling back to a generic parser routine. (By which I don't mean "falling back to treating the stream as text", but rather "falling back to treating the stream as the custom type that the sender specified." Sort of like how, in most languages, you can compile regexes that are available at compile time, but also have a regex interpreter for regexes received at runtime.)
Agreed. And so did the authors of Tcl. They moved away from string representations years ago, but old articles on the previous releases keep being recycled.
What Tcl runtime does internally is irrelevant, on the language level everything is still a string[1]
[1] - Apart from arrays, which are probably the biggest wart in the language. Fortunately, Tcl 8.5 introduced dicts which fulfil the same role as arrays, but as opposed to them, they are strings.
Lispers need to stop thinking that they are the only ones who can do "macros" and "DSLs". The evidence is clearly on the contrarian side. You can have syntax and macros and be practical.
If the failure of Tcl has taught us anything its that performance is important.
>Lispers need to stop thinking that they are the only ones who can do "macros" and "DSLs". The evidence is clearly on the contrarian side.
We know that one can do "macros" in many other languages including of course TCL. The question is how difficult is to do them and how easy is to have control over there. Lisps, due to code being s-expressions, make macro creation extremely easy, and Common Lisp has useful tools to ease working with macros, for example macroexpand, which is integrated into most Lisp IDEs to easily visualize, in-place within the code, how a macro will expand into, step by step.
In comparison to that, most languages that do allow macros, make it a cumbersome, buggy task.
XML, Julia, Rust, Crystal, Forth? support macros too. I was just pointing out that it can be done without sexp. Yes, the tooling sucks. Grammatical implementation is less powerful but that seems better than implementing a grammar ontop of sexps. You get some lexical freedom ontop of pure text that you don't get implementing grammars on top of sexps, respecting lisps tokenisation rules.
I wonder how far Lisp would have come if people had taken the source-source translator approach of js and just implemented an Algol frontend to lisp. There's plenty of things that Lisp had that are far more useful than just macros.
I think Tcl is the Unix answer to Lisp. Worse is better somehow evolved into a Lisp.
> I wonder how far Lisp would have come if people had taken the source-source translator approach of js and just implemented an Algol frontend to lisp.
People did that a number of times over Lisp's history, so Lisp would have come exactly where it has come.
The first Algol-like front end to Lisp was supposed to be the direct descendant of Lisp 1, namely Lisp 2. This branch went nowhere. The descendants of Lisp trace back to Lisp 1.5.
In the early 70's there was something called CGOL. It still exists.
There have been various more recent things in that direction.
And of course, certain contemporary languages are basically sort of Lisp-like things with Algol bolted on.
Tcl wasn't my first language, but it's near and dear to my heart. It's a little oddball for sure.
I can't decide whether it's Python trying to be Bash, or Bash trying to be Python. But it's surprisingly expressive.
Everything is strange about the syntax - from "everything is a string" to "expr" to the weird variable-assignment syntax.
It's also really tiny, and has a great async event-loop framework. It has built-in support for sockets and non-blocking I/O. It has a GUI toolkit that's pretty useful for making quick-and-dirty tools. It's incredibly easy to use interactively with a REPL. And it's still the scripting language of choice for the FPGA industry, which is where I got my start as a software engineer.
The Tcl community is small and stable. And the language just keeps soldiering onwards, in a gradual way. It feels like it's been stable and fully-realized for the last 20 years. No breaking API changes, no syntax changes. It just keeps on keeping on, providing automation to tools that need it. Tcl has outlived a bunch of newer languages, and I hope it sticks around for another 20 years.