Tcl is one of the most underrated programming languages. I have a friend who is working at Intel as tool development engineer (equivalent to software engineer), and Tcl is being used extensively at Intel for chip tooling and design since a very long time.
The modern version of Tcl will be great and a seamless integration with the OS shell will probably the killer niche application. Hopefully Oil Shell can modernize Tcl to some extent and incorporate the best features of Tcl with excellent OS shell integration capabilities.
Tcl's usage at Intel is not particularly special. Perl/Tcl were the workhorses of most (hardware) engineering companies. Back in 2010, a close friend who was working for one such company said that putting Python on your resume won't help you as none of the managers had heard of it. When I joined work around that time, I discovered he wasn't exaggerating much.
While Tcl may be a particularly good language for somethings, well over 90% of the teams I've found using it do so for legacy reasons, and they correlate very highly with poor SW engineers with even poorer SW practices. When I interview for jobs, I steer clear of any that lists Tcl. Not because of the language, but because it's a fairly useful signal about the quality of the team.
All EDA uses tcl, so no surprise Intel would too. A while back I saw some vendors tried to make perl or python an option, but tcl is the de facto language in this space those vendors often abandoned the having multiple scripting interfaces and fell back on tcl. It's quirky, but functional. I like the one line composability
Tcl is deeply rooted in EDA and chip design industry, too many code were written with it. It's like C++, no matter how fancy/modern rust is, there is no way it's going to replace C++ for the next few decades.
One way is to provide Python(python stdlib might be enough for simplicity) gradually, e.g. providing a Python binding for Tcl so for the future, you can write all new piece in Python, and it is either used directly or converted to Tcl behind-the-scene.
python never seemed like a good fit for me, because it's just not suited for DSLs. It's too imperative and has too much built in syntax, you'll end up in callback hell as soon as you need an actually descriptive DSL.
I don't think that opening is there. Tcl may be a terrible scripting language, its usage is not a factor in the value of this or that EDA tool.
If a startup wants to displace an entrenched tool from Synopsys or Cadence, the underlying algorithms (millions of lines of C++ code) must be better. (And once that happens, they'll be acquired by one of these 2.)
I think you are probably right, it is not a make or break factor. Especially when it comes to back-end flows, PPA/TAT/capacity/design closure are the key metrics and the big 2.5 have large moats in these areas.
On the front-end side though, areas like verification, debug, and various lint checks, I think there is more opportunity for smaller players to introduce point tools with better customizability & scripting interfaces.
Ultimately I think the industry is stuck in a local maximum, where the frictional costs of rethinking overall methodology, e.g. from SystemVerilog to Chisel, is too high to justify the change, even if the end result would be marginally (or greatly) better. (Not an endorsement of Chisel, just an example.)
There is Python support in some EDA frameworks, but especially in the digital domain, Tcl is still the gold standard. On the one side this means there are tons of Tcl code around and any experienced engineer is well versed in Tcl.
Also, electrical engineers are usually very focussed on electric engineering. While being being very intelligent, they are often not into programming, so keeping things very simple is an advantage to them.
May be true of analog EEs, but more than half of EEs I work with, and all the good ones, can script well. They use perl/python for day to day, and tcl to interact with tools. The number who can use tcl well is a lot lower than those who can script in general though
I haven't gotten around to looking into Oil Shell. Is it actually drawing any inspiration from Tcl? The "A Tour of the Oil Language" page[1] references Ruby and Python but not Tcl.
Not yet but I've mentioned to Andy (Oil Shell author) regarding the importance of adding command features into Oil Shell and he is actually looking into it [1]. Hopefully he will draw some inspirations from Tcl. Imagine a Cisco like networking command wrapper on top of Linux OS with eBPF backend. It will turn Linux into a ready-made formiddable open source networking OS without the need for NX-OS [2].
If you are looking into a modern version of Tcl, there is TIL [3]. It's a Tcl inspired new scripting language on top of D language. By having D language as the foundation it can perform all the features that you've requested including lambdas, async and then some more [4].
TIL looks interesting, but also both too simple and too complicated. What's the difference between a SubList and a SimpleList? Why does it require SimpleLists instead of using variadic arguments (e.g. why [math ($a + $b)] instead of [math $a + $b])? And why does the author explicitly disallow multiple spaces in a row outside of indentation when this is a purely artificial limitation and does not simplify anything?[1]
But also, why can't I write modules in TIL? It's based on D, sure, but I shouldn't need to write and compile something in D just to be able to have some means of namespacing things. This very much feels like "I want to write most of my stuff in D but I just want to be able to whip up short scripts that for whatever reason I don't want to write in D".
And of course it seems to be ditching the pure simplicity of Tcl. It's like Tcl in that it's a command language and borrows syntax from Tcl, but it seems to be missing the fundamental concepts of Tcl.
[1] The author seems to hate the idea of people lining up equals signs in code or things like that. Ok sure, you can have a style preference. But that's what a code formatter is for, not an artificial and unprecedented level of whitespace significance.
I used a lot of TCL for stuff in the legacy networking space and while the language always seemed to be easy to reason about I seem to remember my main frustration was tracking down syntax errors as runtime problems. Dunno if I was just using the tooling wrong though.
The modern version of Tcl will be great and a seamless integration with the OS shell will probably the killer niche application. Hopefully Oil Shell can modernize Tcl to some extent and incorporate the best features of Tcl with excellent OS shell integration capabilities.