Where Tcl and Tk Went Wrong (2010)
journal.dedasys.com
journal.dedasys.com
In a Lisp, code is trivially parsed because everything is a tree of conses, and the language derives tremendous power from that.
In Tcl, code is trivially parsed because everything is a string, and the language is hamstrung by it; it's a little like trying to code a major project in Bourne shell.
For a time, Tcl had the best C integration story of the scripting languages, with a thoughtfully designed extension interface that was easy to use. But every major scripting language has that now.
Tcl was early to what I guess I'll call mainstream metaprogramming (obviously, Lisp got there first) with "upvar" and "uplevel" and [incr tcl]. But Ruby --- which bears a lot of similarities to Tcl --- did all that stuff much better and more cleanly.
OP> Easily embeddable/extendable.
Easily embeddable language implementations have regrettably remained more uncommon. Sure, extending has gotten relatively easy, but it's still in that "we own the world - you are welcome to visit, as long as you fully respect our partially-documented memory, stack, io, threading, whimsy, and other disciplines" kind of way. And "feature: we're embeddable! (sort of, if you're careful, sometimes, maybe)" toxic hairballs of surprise are still a thing.
- Lua has better support for handling non-trivial data structures, but Tcl interop is a lot less cumbersome, as the host language only needs to know how to pass strings back and forth, basically.
- Tcl is more suitable for writing DSLs due to its homoiconicity, but Lua's more conventional syntax is more suitable for traditional programming.
- Tcl's command-oriented syntax makes it fairly easy to write an expressive shell-like interface for your application (debugging, state inspection, expert interface).
I'll note that via JimTcl [1], Tcl also has an additional niche for bootstrapping software as a much saner replacement for /bin/sh programming, such as in autosetup [2].
But it's taken Ruby a lot of years to get there. And even now, I'd suggest Ruby's `refinements` need more maturing for it to be clear. "uplevel" was wonderfully powerful for creating DSLs, even if rather sharp-edged to develop with.
Type checked, C like, structs, perlisms like if (buf =~ /re/), compiles to tcl byte codes so that L can call tcl and tcl can call L, open source, liberal license.
- not thread safe, it assumes one tcl thread - we stole a bit from the ref count integer to implement "undef" and the tcl core isn't thrilled with that - we default to pcre (but have the old stuff for compat, you can switch at run time) - we broke a bunch of tests (mostly pcre related)
The article mainly examines the corporate and marketing aspects of Tcl's history, and not so much the technical merits of the language. True enough, Tcl lost out to Java and other languages on that account but as the article acknowledges Tcl still has quite a large community keeping it going.
On the technical side, the commentary about "everything is a string" has been extensive discussed and IMO has been shown to be more than not a red herring. In any case many of the "warts" that were prominent in 2010 have been at least partially addressed. Notably in recent years Tcl as acquired many Scheme-like features, an official OOP facility, and coroutines among other "modern" features.
Tcl remains incredibly flexible and adaptable to diverse problem domains. I wouldn't try to write a hardware driver in Tcl, but for many problems it's the most productive tool in the toolbox.
The problems with Tcl start to show up when invariably somebody decides they want something more logically complex than a sequence of commands. Debugging Tcl is extremely difficult, which makes it a huge maintenance liability. For example, a “line number” isn’t much help if the “line” refers to 90% of the file and Tcl thinks all of that is essentially one statement (no matter how many braces are in it). As another example, there are tons of places where a variable is referenced both like "$this" and "this"; that is just a whole series of accidents waiting to happen, especially when you consider that many functions depend on argument order and they don’t necessarily produce obvious malfunctions when variables are misused.
I think this may have improved since you last looked:
$ cat j
#!/usr/bin/env tclsh
puts [ set a [ expr { 9 + [set x]}]]
$./j
can't read "x": no such variable
while executing
"set x"
invoked from within
"expr { 9 + [set x]}"
invoked from within
"set a [ expr { 9 + [set x]}]"
invoked from within
"puts [ set a [ expr { 9 + [set x]}]]"
(file "./j" line 3)
$this is essentially [ set this ], returning its value, or deferencing the value, if you will. The entire set of rules that Tcl abides by are contained entirely in a small document (here [0], with commentary). It's really not difficult to figure out.
Also, I have no problem with the logical "$x" syntax; rather, the fundamental flaw is that the string "x" could be used to change the variable "x" without any clue in the syntax that this may happen (e.g. no "&x" like in C, no "\$x" like in Perl; just "x", or worse "$y" that resolves to "x"). And don’t even get me started on multi-level "upvar". This whole thing is a minefield, especially if people use simple variable names that can’t be searched in a code base. If you’re really unlucky, you accidentally swap the order of parameters and it just so happens that your "$x" value is almost plausible so you never notice the disaster you just created. It’s simply a “write once, modify never” coding style: it saves the programmer 10 seconds and creates the most brittle of code. I would never recommend it to anyone.
Used to work on TCl around 10 years back, I dont remember much but I do remember that debugging it was difficult or maybe I was not so smart.
When I used Python I quickly loved it because atleast I could debug any issues quickly.
And "expect", which is the only reason I've ever touched TCL in my professional career.
Eggdrop was my first introduction to Tcl/Tk and though the learning curve was harder than Perl, it was fascinating.
There was quite a lot to like about Tcl. Mainly that everything just worked.
Packages were solid, getting stuff off the web was easy.
It's a nice language. Try it.
I have no real need for it these days, but I'm happy to see it is still in use even if it's not as popular as it once was.
The interpretation and the braces in braces parts are annoying. Not very readable.
Modularization is not easy and other files can't be imported as fast as with Python or ES2015.
Resources and tutorials are not readily available, there is no community I ran into to ask stuff compared to the rest of the language world.
Finding a test framework or linting mechanisms is not easy either.
I'm no friend of Tcl at the moment, I'm not sure if that will change.
Languages don't really seem to "come back" that much. At some point being for too long out of the mainstream weighs against you.
Objective-C came back from obscurity with a vengeance.
I think I know all the major web resources for the language, and I agree. In terms of learning material available online Tcl is simply very much behind Perl, Python, and Ruby. If you're learning Tcl from scratch today, I'd recommend to start by following https://learnxinyminutes.com/docs/tcl/ (ignore the broken syntax highlighting) and then https://www.tcl.tk/man/tcl8.5/tutorial/tcltutorial.html, and playing with the REPL extenstively (I am a fan of using http://homepages.laas.fr/mallet/soft/shell/eltclsh as your REPL for how it does completion). If you want a book, there are quite a few, but most of them outdated. While I haven't read it myself, I have heard good things about the most recent, The Tcl Programming Language, and I enjoyed its author's other writing on Tcl. The man pages that Tcl comes with are a high quality reference for its commands. The Tcler's Wiki (https://tcl.wiki/) is also indispensable as a reference and a starting point for research.
>there is no community I ran into to ask stuff
I highly recommend the "tcl" tag on StackOverflow and #tcl on Freenode. The former has some seriously dedicated people answering StackOverflow-scoped questions. The latter is where the developers of the language hang out and suitable for more open-ended discussion.
>Finding a test framework
The standard for testing Tcl code is tcltest (manual: https://www.tcl.tk/man/tcl/TclCmd/tcltest.htm). The best examples to learn from are in Tcllib, the language's standard library, e.g., https://core.tcl.tk/tcllib/artifact/47f195f05f695703. If you need a framework for functional testing, try Caius — http://caiusproject.com/.
>linting mechanisms
There are several linters for Tcl; see the list at https://tcl.wiki/3162. At one point I tried using ttclcheck because it implements static type checking, but never stuck with it or any other linter. This may be due to Tcl's sparse syntax and highly dynamic nature, because I do use linters and style checkers for other languages.
For strings, if a null was in the data, it truncated at that point, and there wasn't an alternative type that could handle it.
And converting a strong reference to a string would give you a weak reference. So you wouldn't be able to, say, convert a list that included a strong reference to a string, then convert the string back to list, and have exactly the same list that you started with because the strong reference in the original list would be a weak reference in the resulting list.
Strings can represent list and dictionary data structures, and combinations of, but I'm not sure that strings represent all Tcl data structures in the same way. I'm thinking of Tcloo and Tcl arrays - I don't use those and know little about them.
If anyone is interested in seeing a recent app example, check out git-gui [0]. It's bundled with git.
I'm on macOS. Installing the latest version was confusing. There's a tcl-tk package in homebrew-core, and a tcl package in homebrew-cask. The tcl package is ActiveTcl. What's the difference? On the tcl.tk site, if you try getting the binary it'll point you to ActiveTcl.
Diving into the wiki was very confusing. Lots of articles have comments by random people, and it's not always clear if their comments are relevant or outdated. There's dozens of links, and hard to discern which ones are worth looking into. Lot of links are dead.
The differences between the different Tclkits is not obvious, too many choices. And how do they relate to starkit? I think I remember having downloaded an sdx compiled widget, and I couldn't find the sdx utility, so I couldn't unwrap it.
Tons of dead links in the Great Unified Tcl/Tk Extension Repository [0]. Almost nobody puts up screenshots of their widgets.
It's also frustrating to find out that Tcl Dev Kit is a paid component. I'm probably spoiled by other languages, so this might be pretty entitled of me, but I'd consider the dev kit's tools to be essential. The most important tools for me would be the debugger, linter, and inspector.
As a standalone user, I'm not going to pay $300 just to play around with a programming language. It would be a huge improvement if they were made freely available, even if under a non-commercial license. Then people can try it out, and if they find a good use-case in their job, they might buy a license. I've done that with other software. To give a concrete example: in a few college projects and hackathons I used highcharts [1], a JS charting library. Since it provides a free non-commercial license, I ended up learning how to use it pretty well. Fast forward a bit, and the company I was with wanted data-dense interactive charts. I evaluated a few options, but was most satisfied with highcharts, so I happily convinced them to buy a license.
That's also similar to what Qt has been doing too, and it has been hugely successful. You can use the free open source version, which is licensed under GPL and LGPGv3. But if you want to keep things locked down, you can pay ~$3500/year for their commercial offering. Most humans probably won't pay for the commercial version unless they're full time app developers, but businesses tend to have much deeper pockets.
Expect lets you automatize user interaction with terminal applications. And then there is DejaGnu, a testing framework, that uses Expect and Tcl. Probably the most important software project using DejaGnu right now is the GNU Compiler Collection. A not so significative project that also uses it is the interpreter of my toy language :) it took me a little bit to understand how to integrate it with Autotools but now I really like it (here is an example: https://github.com/pera/ad-hoc/blob/master/testsuite/ahci/dg...).
For me they went wrong on being a pure interpreted language, having to write all performance critical parts in C wasn't scaling for us.
It was there that I learned only to use languages with JIT/AOT support.
Also having Tk stuck on a Motif look and feel wasn't appealing to us as a mechanism to deliver good UI/UX to our customers, specially to the majority of them on Windows.
Read about it in Ousterhout's Dichotomy: http://wiki.tcl.tk/9865
Besides Python
- Sun's pushing Java and killing everything else
- Stallman's distaste for anything non-GPL
EDIT: wording
and it didn't have the ecosystem to justify it
I think Tcl fails on the basics, nothing too deep, subtle or obscure
But it was an awful language because it was so limited. Many commands in DC naturally want to return objects. But those don't exist in Tcl (maybe they do now?).
So DC had to invent "collections" to make up for the limitations. Which confused the heck out of engineers. Resulting in Stack Overflow questions such as this one:
https://stackoverflow.com/questions/17864980/collections-to-...
So, yeah, EDA uses Tcl but its for historical reasons rather than because its a good tool for that purpose.
"One actual technical problem that Tcl faces is the concept that all values must be representable as a string. This is more or less ok for things like lists, hash tables or numbers, but is problematic when the user wishes to represent a value that simply isn’t a string."
Tcl is cool because you can manipulate the string form of values incredibly easily, but people confused this with "Everything is a string". It should be "Everything can be manipulated like a string".
The only language that I think will escape this is JS due to the browser - unless we make a new sandbox app container designed to run scripts (a la java plugin).
https://www.destroyallsoftware.com/talks/the-birth-and-death...
Quirky bit of personal history that amuses me to no end.
1 1 Java 12.961% -6.05%
2 2 C 6.477% -4.83%
3 3 C++ 5.550% -0.25%
4 4 C# 4.195% -0.71%
5 5 Python 3.692% -0.71%
6 8 change Visual Basic .NET 2.569% +0.05%
7 6 change PHP 2.293% -0.88%
8 7 change JavaScript 2.098% -0.61%
9 9 Perl 1.995% -0.52%
10 12 change Ruby
Python, BDFL
PHP, BDFL
Perl, BDFL
Ruby, BDFL
The rest (even though you wouldn't compare them to TCL cuz they're not in the same niche) have/had huge corporate sponsors and/or are entrenched industry standards and have also had iconic/notable figureheads (Gosling, K&R, Stroustrup, Hejlsberg, Eich).
- lua appeared 2 years before
- python 5 years before
- perl 7 years
Instead we have this monstrosity with no namespace, terrible error handling, pitiful stdlib, limited expressiveness, atrocious data structure and type handling, so full of traps and gotchas we had to write a book about the part of the languages not to use.
Also, given that DSSSL was an early contender against CSS (and a great deal more powerful), we could have basically had Scheme there too. https://blog.cloudflare.com/the-languages-which-almost-becam...
Lastly, because S-expressions can represent everything that is representable in SGML/XML/(X)HTML, but more succinctly, we could have basically had Scheme instead of HTML.
Imagine if, instead of JS/HTML/CSS, we had... Scheme. I'm pretty sure that the alter universes where this happened also all have peace, warp drives, and working cold fusion.
The worst crime Sun ever committed will turn out to be influencing the birth of JS.
I suggest (along with David Moon (Lisp Machine)) that a key disadvantage of sexp with respect to *ML, is that there's no place to hang on "extra stuff". Many lisps allow you to decorate the nodes with properties once the nodes are read, but the properties lack a printed representation. It's a homoiconicity fail. Sort of like not allowing comments in JSON might be, if a common case was then to attach comments to the objects after they're loaded. That very isn't common, so the restrictiveness is worth it. It's not clear that's the case with sexp.
"In 1993, the only real contender was Tcl, which had been explicitly designed to be embedded into applications. However, Tcl had unfamiliar syntax, did not offer good support for data description, and ran only on Unix platforms. We did not consider LISP or Scheme because of their unfriendly syntax. Python was still in its infancy. In the free, do-it-yourself atmosphere that then reigned in Tecgraf, it was quite natural that we should try to develop our own scripting language ... Because many potential users of the language were not professional programmers, the language should avoid cryptic syntax and semantics. The implementation of the new language should be highly portable, because Tecgraf's clients had a very diverse collection of computer platforms. Finally, since we expected that other Tecgraf products would also need to embed a scripting language, the new language should follow the example of SOL and be provided as a library with a C API."
https://news.ycombinator.com/item?id=1905155
Do you actually use JS? What's with the royal "we" that only His Majesty King Crockford (I'm friendly with Doug) can use?
For all that JS may suck, I somehow think anything early 2000s MS would have pushed would have been worse. VBscript, perhaps?
The real problem would have been that they would have had to give other browsers a chance to adopt the new language, possibly even by providing dedicated resources to implement it for them; but Ballmer's MS didn't really "do" cooperation, they were happy to push ActiveX to lock out competitors.
Still, they did have the chance.
https://devchat.tv/js-jabber/124-jsj-the-origin-of-javascrip... (transcript at bottom)
https://brendaneich.com/2012/04/the-infernal-semicolon/#comm...
Netscape management gave "make it look like Java" orders. There was no time, on top of this. Last (more below in another reply) no extant command line interpreter was ready to be put in a browser. Way too many non-portable OS dependencies, unsafe FFI, etc. It was JS or bust (aka VBScript).
I don't think it's the same use case. Tcl is really good as a CLI language, like Python.
NodeJS is not very good when it comes to test stuff in the CLI quickly due to its async API. Even when there are synchronous versions, most libraries are async so wrapping everything in promises is a bit of a pain.
IMHO Tcl really lost to Python, not JS. Python is an excellent way to test ideas quickly on the CLI.
I do a lot of web-scrapping and data transformations and to be able just to open a CLI, call a lib and test things out in 5 minutes for a PoC is just RAD.
Not saying you should change languages though, especially if you're happy with your existing toolset! But if you feel up to it, you might be surprised upon giving it a try.
I've written a few one-off scrappers without any complaints. It also works well for simple cross-platform scripts.
I don't recall the developer experience when installing python on windows, so maybe it's just as good. With node you only have to download and install the official release, and everything is good to go. Even when some package has native code, it generally "just works".
Python was a killer of both... it wasn't as hostile and freaky as Tcl and arcane as Perl.
That's not my recollection. And I disagree with the fashion thesis too. :)
So I used Tcl, perl, and python in the early 90's. Tcl owned embedding and Tk. Perl owned sysadmin and CPAN. And was fast. Python owned OO and "nice professionalism". Perl and python had hashes. Tcl got hashes. Tcl and perl picked up OO. Perl and python mooched Tk. But Tcl remained furry and batteries not included. Perl remained buggy and weird and unprofessional. And python remained without good code sharing. Turns out people wanted "nice" more than embeddability or CPAN. All three communities knew what their strengths and weaknesses were. And were variously able to deal with the weaknesses, or not.
Today some of main Python libraries are in C, specially the ones that need speed, like numpy, pandas and all the Data Science eco-system.
Desktop user interfaces became less important with the Web.
Python also got a good regexp engine. Its legibility, batteries included and interactive shell beat them all.
Ironically SWIG originated on Tcl, IIRC
So I agree the field has remained disorganized, with it's research and development severely underfunded. There is no language implementation wiki. The field's research is often by years of intermittent incremental papers, and little bus-factor one projects. We're down to like one biggish industrial research lab. And ARPA, sigh. And professional progress, often seems sort of exploration by an army of wandering ants, rather than purposeful pursuit of economic development and human progress.
So with the poorly organized professional knowledge, and the unacceptability of high onboarding costs, yes, there's a lot of random motion. And the scrambling mob carves a flood plain.
But.
Cohort learning is actually a good learning strategy. And given how big the field has gotten, and the criticality of community, it's a plausible local heuristic to "follow the mob - it's path will likely at least be plausible, even though not optimum, and the inevitable obstacles can be collaborative tunneled through, or filled in with bodies". And the profession is not just larger, but vastly better trained than it used to be. There's a massive education process in progress.
And the "driven by fashion" argument gets used a lot. At least sometimes, it's clearly an "argument from failure of imagination". A "I can't imagine what they're thinking, they must all be nuts". Sometimes your expertise in some domain corner is enough to confidently say that. Or the nuttiness is nicely compact - "Google backs it, so I'm sure it will be supported for decades". But it seems most use of the argument is more... ambitious than that.
> Tcl is a powerful language that you can use to create complex systems with simple code, the holy grail.
Tcl/Forth/Smalltalk/Lisp/Self/APL/Oz/... we've had many many languages with glints of grail. We're just still really bad at combining and polishing them.
There's little funding for it. So it's a multi-decade incremental creep. An "if you're still alive to see it" bright side might be, it looks like the transition to "no longer crippling", when it finally happens, might be a "boom - new world" kind of thing.
EDIT: aside: I saw the other day that React native(?) could now be packaged up with it's own little embedded faux file system... and thought, yay, we're catching up with 1990's Tcl! (The Tcl app distribution story was excellent). You just need to have a half century of patience, and it will all work out. Patents only last two decades. And we're only failing, what, 7 billion people? Delay a year, miss a 130 million person cohort. No problem.