I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones. I love how easily you can make DSLs.
I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method. Give me a stdlib that offers more functional stuff. Heck, give me async/await. And more too.
But even without all of that, Tcl is still a wonderful language and I often lament how it’s mostly died out.
[1] There’s at least one library I’ve used that actually does this, but AFAIK there’s no way to do this in pure Tcl so it must be a feature of the C API. Also I’m not sure if it actually tracks the object or just the variable it’s assigned to.
Tcl is one of the most underrated programming languages. I have a friend who is working at Intel as tool development engineer (equivalent to software engineer), and Tcl is being used extensively at Intel for chip tooling and design since a very long time.
The modern version of Tcl will be great and a seamless integration with the OS shell will probably the killer niche application. Hopefully Oil Shell can modernize Tcl to some extent and incorporate the best features of Tcl with excellent OS shell integration capabilities.
Tcl's usage at Intel is not particularly special. Perl/Tcl were the workhorses of most (hardware) engineering companies. Back in 2010, a close friend who was working for one such company said that putting Python on your resume won't help you as none of the managers had heard of it. When I joined work around that time, I discovered he wasn't exaggerating much.
While Tcl may be a particularly good language for somethings, well over 90% of the teams I've found using it do so for legacy reasons, and they correlate very highly with poor SW engineers with even poorer SW practices. When I interview for jobs, I steer clear of any that lists Tcl. Not because of the language, but because it's a fairly useful signal about the quality of the team.
All EDA uses tcl, so no surprise Intel would too. A while back I saw some vendors tried to make perl or python an option, but tcl is the de facto language in this space those vendors often abandoned the having multiple scripting interfaces and fell back on tcl. It's quirky, but functional. I like the one line composability
Tcl is deeply rooted in EDA and chip design industry, too many code were written with it. It's like C++, no matter how fancy/modern rust is, there is no way it's going to replace C++ for the next few decades.
One way is to provide Python(python stdlib might be enough for simplicity) gradually, e.g. providing a Python binding for Tcl so for the future, you can write all new piece in Python, and it is either used directly or converted to Tcl behind-the-scene.
python never seemed like a good fit for me, because it's just not suited for DSLs. It's too imperative and has too much built in syntax, you'll end up in callback hell as soon as you need an actually descriptive DSL.
I don't think that opening is there. Tcl may be a terrible scripting language, its usage is not a factor in the value of this or that EDA tool.
If a startup wants to displace an entrenched tool from Synopsys or Cadence, the underlying algorithms (millions of lines of C++ code) must be better. (And once that happens, they'll be acquired by one of these 2.)
I think you are probably right, it is not a make or break factor. Especially when it comes to back-end flows, PPA/TAT/capacity/design closure are the key metrics and the big 2.5 have large moats in these areas.
On the front-end side though, areas like verification, debug, and various lint checks, I think there is more opportunity for smaller players to introduce point tools with better customizability & scripting interfaces.
Ultimately I think the industry is stuck in a local maximum, where the frictional costs of rethinking overall methodology, e.g. from SystemVerilog to Chisel, is too high to justify the change, even if the end result would be marginally (or greatly) better. (Not an endorsement of Chisel, just an example.)
There is Python support in some EDA frameworks, but especially in the digital domain, Tcl is still the gold standard. On the one side this means there are tons of Tcl code around and any experienced engineer is well versed in Tcl.
Also, electrical engineers are usually very focussed on electric engineering. While being being very intelligent, they are often not into programming, so keeping things very simple is an advantage to them.
May be true of analog EEs, but more than half of EEs I work with, and all the good ones, can script well. They use perl/python for day to day, and tcl to interact with tools. The number who can use tcl well is a lot lower than those who can script in general though
I haven't gotten around to looking into Oil Shell. Is it actually drawing any inspiration from Tcl? The "A Tour of the Oil Language" page[1] references Ruby and Python but not Tcl.
Not yet but I've mentioned to Andy (Oil Shell author) regarding the importance of adding command features into Oil Shell and he is actually looking into it [1]. Hopefully he will draw some inspirations from Tcl. Imagine a Cisco like networking command wrapper on top of Linux OS with eBPF backend. It will turn Linux into a ready-made formiddable open source networking OS without the need for NX-OS [2].
If you are looking into a modern version of Tcl, there is TIL [3]. It's a Tcl inspired new scripting language on top of D language. By having D language as the foundation it can perform all the features that you've requested including lambdas, async and then some more [4].
TIL looks interesting, but also both too simple and too complicated. What's the difference between a SubList and a SimpleList? Why does it require SimpleLists instead of using variadic arguments (e.g. why [math ($a + $b)] instead of [math $a + $b])? And why does the author explicitly disallow multiple spaces in a row outside of indentation when this is a purely artificial limitation and does not simplify anything?[1]
But also, why can't I write modules in TIL? It's based on D, sure, but I shouldn't need to write and compile something in D just to be able to have some means of namespacing things. This very much feels like "I want to write most of my stuff in D but I just want to be able to whip up short scripts that for whatever reason I don't want to write in D".
And of course it seems to be ditching the pure simplicity of Tcl. It's like Tcl in that it's a command language and borrows syntax from Tcl, but it seems to be missing the fundamental concepts of Tcl.
[1] The author seems to hate the idea of people lining up equals signs in code or things like that. Ok sure, you can have a style preference. But that's what a code formatter is for, not an artificial and unprecedented level of whitespace significance.
I used a lot of TCL for stuff in the legacy networking space and while the language always seemed to be easy to reason about I seem to remember my main frustration was tracking down syntax errors as runtime problems. Dunno if I was just using the tooling wrong though.
You can't really get full-fledged closures without ditching the fundamental concept of Tcl, which is that "everything is a string". It's the same problem as OOP, or rather, object lifetime management - you can have that in Tcl, but only as long as you manage the lifetimes manually. For closures, this negates most of the benefits they provide.
It would work if they basically copy the current value of the identifiers (i.e. by-value, not by-reference). Or by capturing the procedure context. You can synthesize the variable capture by hand but that would indeed force everything into a string, which is a potential performance issue and would also break any sort of automatic lifetime management.
Even just making it easier to have lambdas reference variables from their creation scope while the creation scope is still on the stack would be an improvement. Today you can use [info level] and then construct a lambda body that uses [upvar #$level …] to reference variables from the original scope, but it's a PITA. Or you could cram the level into a temporary namespace variable to avoid having to manually construct a body string, but it's still a PITA.
The basic problem is, how do you even know that something is a variable reference? Remember, everything is a string. This means that the body of the lambda is also a string. Now you can require that it's an eval'able string at the point of creation, and parse it to identify any $var references. But you can only do that for the topmost level of the lambda, because it can contain nested strings - and you don't know which of those strings are code, and which aren't (since it's really the command to which the string is passed that decides to treat something as code or not).
Furthermore, if you process $ like that, this won't work for any other command that takes a variable name as an argument, and does something to it.
"upvar" is explicit, there's no guesswork involved there. But how would you handle a lambda with a body like this?
if {$foo == 1} { puts $bar } else { puts $baz }
given that all the {}s are just non-expanded string literals, and what makes them be interpreted as code is the implementation of "if"? Sure, you can hardcode "if"; but the whole point of Tcl is that syntax constructs like that can be easily implemented as a library, which breaks if you have to special-case them in lambdas for them to work.
Now if you used upvar, this kinda sorta works, but only so long as the lambda is immediately invoked by the function that it was passed to. If you want to pass it on, you'd have to wrap it in another lambda. And, of course, this only works for lambdas that don't escape.
[list apply {args body} $value1 $value2 ...] can be used to capture values without actually generating a string because even though conceptually a list is a string, it's actually stored internally as a list.
> I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method.
If you take a look at jimtcl (minimalistic reimplementation of tcl, started by antirez - he of redis fame), it did take some steps towards the above.
- unified arrays with dicts
- proper lambdas
- (sort of) closures
In many ways, I personally feel its a shame that jimtcl has felt the need to tie itself down to tcl-compatibility as much as it has. A few more warts could have been fixed along the way.
Anyway, check it out. It's manpage is quite comprehensive
Re [1]: the libraries that delete an object when it goes out of scope use Tcl's built-in "trace" command, which lets you execute code when a command terminates or when a variable is deleted, among other events. So no, it's all done in pure Tcl.
Oh it looks like I misremembered. I thought the API I had used was something like `set doc [foo parse $input]` and the document would go away when the variable does. I found it, it's the tDOM library, and it was actually `dom parse $xml doc` that does the automatic freeing, whereas `set doc [dom parse $xml]` requires manual deletion. So it probably is using trace.
I believe a lot of what makes Tcl the way it is is precisely because it does things different from most other languages with regards to closures, lambdas, scoping etc.
If Tcl had the other stuff, would it need upvar/uplevel or its funny braces and substitution rules? I'm no expert at Tcl but when I learnt it (mainly to use tkinter) I really had to rewire my brain to think about something as simple as _variables_ so differently
I love the language as well, we did lots of cool stuff with Tcl in 1998 - 2003, we were already doing our own Rails, but a tiny startup in Lisbon does not rock the world the same way.
However that experience lived on and gave birth to OutSystems.
Imagine the timeline where Tcl, Scheme, xlisp, or any of the dozen other existing, mature, available, non-turrible programming languages was chosen for DHTML.
Versus the poorly conceived, poorly implemented, from scratch rush job based on a fundamental misunderstanding of Self.
In fact, we could have ended up with Self. Imagine that timeline.
>I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
I'm not sure that's an appropriate phrasing. The word "choice" implies that alternatives are available, and there a decision was part of the selection. I think "Has been used for the language that momentum has demanded" might be more accurate. ;)
Synopsys used to have their own scripting language that was so limited you couldn't define functions. You had to resort to setting variables as pretend arguments and continually including another script for any code you needed to invoke multiple times without duplication.
I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
Have you ever had to debug code written by someone who implemented their own control flow operators? Not fun. I've seen it done with C macros. Don't do that.
A poor choice in C indeed, but languages like Haskell or (it seems, don't know much about it myself) TCL are practically made for it. Debugging "own control flow operators" is nothing very special there, mostly because control flow operators themselves aren't very special in those languages.
From another way to look at it, nobody seems to bat an eye when using things like items.forEach() in, say, Node.js, even if this sort of thing is arguably much uglier to do in JavaScript vs. a "first class" functional language where common control flow operators are often just plain functions themselves.
Dynamic scoping (`local $foo`) is nothing special in Perl. It's also a construct which, like novel flow control operators, by its nature leads to code which is complicated to reason about and difficult to debug.
There are many programming language concepts which may be innovative and beautiful, but whose popularity wanes because of practical flaws. (EIAS is another.)
There is a world of difference between writing a control structure in C using macros... and Tcl. And even more so when you look at a proper Lisp. Being able to construct new control structures is amazing; I miss it when in other languages.
I totally got turned around on TCL. I hated it when I started as I was mostly programming in c#, but now after year of sporadic use I absolutely love it for it's simplicity when manipulating text files and strings.
Checkout Scala. It allows you to do pretty much the same, i.e. the syntax is so flexible that you can even make 'puts "Hello World!"' work and implement your own "if" that looks like the builtin one. It also has first-class lambdas and the other stuff that you asked for, but obviously there are also differences.
Scala runs on the JVM, which makes it not at all suited for scripting. Scala also seems to have something of a racism/white supremacy/misogyny problem.
> Scala runs on the JVM, which makes it not at all suited for scripting
The JVM is one target. You can also generate and run javascript code or create a native binary. But it will not be as smooth as e.g. python, so I can't really recommend that.
> Scala also seems to have something of a racism/white supremacy/misogyny problem.
I'm a member of that community for a long time and I really don't think so. I always thought the Scala community was a rather welcoming one. There certainly have been incidents in this community, just like they have been in other communities, but I don't think "Scala" has more of a problem here than any other language.
I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method. Give me a stdlib that offers more functional stuff. Heck, give me async/await. And more too.
But even without all of that, Tcl is still a wonderful language and I often lament how it’s mostly died out.
[1] There’s at least one library I’ve used that actually does this, but AFAIK there’s no way to do this in pure Tcl so it must be a feature of the C API. Also I’m not sure if it actually tracks the object or just the variable it’s assigned to.