Where Tcl and Tk Went Wrong
journal.dedasys.com
journal.dedasys.com
What does Tcl offer? Half this article is about Tk, but Tk is (a) a generic GUI toolkit, and therefore competing directly against everything from Java Swing to Flash to (the big winner) HTML/JS/CSS. And (b) not Tcl-specific, right? Didn't I see Perl bindings for Tk go by?
As an embeddable language Tcl somehow got eclipsed in the branding battle by Lua (don't ask me why; not my field, all I can do is repeat glib rumors) and in web-app land it got crushed by PHP just like Perl did. If you want to toy with extensible syntax in your language you can use Ruby DSLs; if you want the real thing you go to a Lisp. Tcl's a lousy language for mathematics and I can't imagine replacing a shell script with it (I think I vaguely remember a Tcl shell, but maybe I dreamed that, because it has no mindshare). It doesn't run in the browser. So what is Tcl for? Just tell me in a sentence. And, no, "being a nice average general-purpose language" is not a brand worth anything.
1) Ruby DSL, really? After you understand Tcl you'll discover how Ruby is not really a language suited for DSLs compared to TCL, even if Ruby is quite suited for this use case indeed. Tcl is not just much more DSL-ish as a syntax and structure, but also has some kind of "run time lisp macros" (uplevel) that let you do everything in your DSL.
2) Tcl built-in event loop provided a way to write very fast evented I/O apps easily. Something node.js is providing now was, in a different form, available since 10 years, and nobody noticed. However it is still very interesting, now with support for coroutines Tcl is very suited to take the state between firing of events.
3) Tcl as alternative of shell script is great, for a number of reasons: from the [glob] command to an [exec] command that is able to mimic a lot of what a shell can do. See the Redis unit test for an example of scripting stuff with Tcl.
4) Tcl vs Lua: Lua IMHO wins in the implementation, Tcl has not an ANSI target, and is a much larger implementation. But actually it is possible to write a large subset of Tcl in a very small space. I tried it in the past with http://jim.tcl.tk, and indeed now this implementation is in use in the embedded world.
Tcl lost at this point, but programmers willing to understand it will do a great thing, it is like to be exposed to a different new world. It's like learning Scheme or FORTH: mind changing.
I admit I've forgotten most of my Tcl, though. Because I haven't had a compelling need to use it in ten years, so I haven't. See above.
Tcl is indeed "a different new world, like learning Scheme" (though Scheme has SICP, and Tcl has... what?) but this essay is not about "why Tcl makes a great academic paper" or "why Tcl is a great way to spend a Saturday night" but rather "why Tcl is not popular as a mainstream programming language". Being "mind-blowing" is not only not a path to mainstream popularity, but it's also a field with a lot of stiff competition: Scheme, Forth, Factor, Erlang, Io, Common Lisp, Haskell... several of these are on my personal todo list ahead of revisiting Tcl.
I ported SimCity to Unix with TCL/Tk around 1993 or so, and this is the typical code for handling an async IRC chat connection -- look at all the globals and spaghetti code: http://code.google.com/p/micropolis/source/browse/trunk/micr...
Without closures, it's even harder to write reusable "components" that you can make more than one instance of. Look over that TCL code and try to figure out how it handles making more than one instance of each kind of window. It supports multi-player mode by making multiple X11 server connections, so it can have more than one instance of each kind of window, one for each X11 server connection, and some windows can have multiple instances on each server (like the edit and map windows).
I had to write a lot of boilerplate infrastructure for creating/destroying windows and handling events, to store all the state for each window in global variables and structures, keyed by the window id, linked together symbolically, so windows on different screens don't clash, and even fix some bugs in Tk itself relating to having multiple windows on different servers: http://code.google.com/p/micropolis/source/browse/trunk/micr...
To create each window it would reload a file like this each time that set up all the globals and window links and bindings like "LinkWindow $head.map $win" and "bind $win <Visibility> {[WindowLink %W.view] Visible [string compare %s FullyObscured]}". Ugly shit man! Give me closures any day: http://code.google.com/p/micropolis/source/browse/trunk/micr...
Multi-head X11 support was not well tested in 1993, it got confused if the X11 servers had different visuals, and the TCL menu handling code had to be rewritten to keep the state in per-window structures instead of globals that made the assumption that only one menu could be popped up across all screens at once. Notice the "set screen [winfo screen $w]" and "set tk_priv(posted@$screen) $w" stuff: http://code.google.com/p/micropolis/source/browse/trunk/micr...
The thing TCL doesn't have are pipes. Not even Unix pipes, just some way of chaining little blocks together, so that complexity extends rightward, as you type.
exec cat /etc/passwd | grep root
You can also use Tcl's open command with a pipe to do back and forth communication with a subprocess.
I don't think it's the hook that's missing (Tk was a major hook; if the hook was all that mattered, Tk would have kept up with the times).
I think it's just that Tcl is a crappy language. It's always love with me and Tcl, but like this article says: everything in Tcl is a string. The "language" is just a stream of commands --- it parses comments! For normal programming tasks, it has all the syntactical downsides of Lisp notation, but none of the upside. It's slow.
There was a time when Tcl was killer because it was the easiest language to (a) embed and (b) extend in C. But lots of languages do that well now, and Lua specializes in it. Embeddability was the last hook for Tcl. It's hard to see why anyone would choose it for a system today.
set foo [+ 1 2]
incr foo 3
puts $foo
I can and have. Whenever a shell script starts getting at all complex, I grab a scripting language. These days it's Ruby, but in the past it was Tcl, and it did a fine job: it has a bunch of built in commands that are convenient for system tasks, and plays very nicely with the environment it lives in. So, quite the contrary: due to the OO (although this is being fixed) and 'GC/reference' problems, I'd be more concerned about big systems in Tcl, and not at all with little scripts, where it still is an excellent choice.
You're quite right about "the web" being the big winner in terms of "cross-platform toolkits" and as a bonus deployment is also way easier. Tcl can do the web though. See: http://flightaware.com/ . But of course in that field, things didn't play out like they did with Tk, where it was probably the most popular option.
Also, while I use Ruby these days quite happily, Tcl has it beat in terms of DSL's, IMO, due to the extreme simplicity/plasticity of the syntax.
Why Lua (probably deservedly) started beating Tcl in the 'embedding' space is covered in the article.
Python, by the way, does pretty well with being a "nice average general-purpose language", although I do sense some envy on their part over the popularity of things like Ruby on Rails. So I do and I don't agree with your conclusion. If Tcl were where Python is today, it would be a much healthier language, even if not the biggest thing out there. Also, by having a 'reservoir' of people using a language, you're more likely to get someone doing something that turns into a big hit.
tcl> foo { bar baz zot }
The only way to use the result of the command is to surround it with square brackets, which requires me to jump all over the line:
tcl> quux [foo { bar baz zot }]
This makes it pretty difficult on a readline-style interface to cobble together small commands Unix-style.
And while interactive languages may be a small niche today, I think part of the reason is that nobody's solved the usability problems as elegantly as the Unix shell has.
There was (is?) a plugin for all major browsers. It's interesting that in html 4.01 spec Tcl was in examples alongside js and vb: http://www.w3.org/TR/1999/REC-html401-19991224/interact/scri...
Unfortunately, it's too simple, in one fundamental way which makes the language very flawed: it does not have closures. Callbacks (which are fundamental to I/O or GUI work) are executed from a scope which has access to nothing but global variables. As a result, users of the language must either stash everything in globals, or else perform awkward explicit variable substitution, taking care to distinguish between variables to be evaluated in the present context and variables to be evaluated in the callback context and escape them appropriately (like writing a Lisp macro, but this is in every callback.)
And if you want to implement any higher-order functions, say filter by predicate, you have a problem because you want to pass an argument to the predicate, but you also want the predicate to have access to variables in the scope where it was defined. The predicate is not a first-class function: it's a block of code that you can eval. In fact, you can eval it anywhere on the call stack. That's right: Tcl gets the job done using functions which evaluate blocks of code in their caller's context. If that sounds scary, good, it is.
Exhibits A and B: PHP and Java.
It's just that solving a complex problem with a simple tool can often be more complex than solving it with a complex tool, even if the complex tool requires a bit more up-front learning.
Working with Tcl makes it very clear that spending some of your complexity budget on closures pays off big.
As far as what makes for a popular language, clearly corporate support is the key factor in some cases, good libraries in others, but I can't really explain the rise of Python except by its design.
The author talks about the ease with which the language's syntax can be extended. For me, this gives me the same feeling I get when I'm using Lisp. I'm aware of the large differences between the languages, but the feeling of freedom they both provide makes using them flat-out fun.
I guess that's really the key to allowing easy syntax extensions; it means that programs are represented in the very core data type, and you have lots of tool to manipulate it.
Although it was embedded in our browser (IXI Mosaic) as standard, Netscape was formed at the same time, got huge market share and did their own thing. But TCL could have been a contender ...
I personally switched from TCL to Python in 2000. The lack of a standard way of keeping data and the methods that operate on it together ("object orientation") made larger projects too clumsy.
It's like Larry Wall would write a book about Django.
I think the author is right that it was a mistake to ignore Linux. It didn't have the desktop user marketshare, but it did have the developers that were doing a lot of the work on open source code.
Even at that time (just before GTK+ 2.0, around 2001), expected toolkit features had jumped ahead of Tk quite a bit; in porting the widget I think I added model-view separation, pixel-based (vs. line-based) scrolling, international text layout, input methods, accessibility, lots of optimizations, support for non-XPM images... likely more I can't remember.
Tk lacked all that stuff, and various GTK-related companies and Troll Tech (Qt) were investing in building all of it for GTK and Qt. That led to a situation where only those two toolkits, that had been modernized, were acceptable to lots of decision-makers who were deciding what to ship with Linux distributions or GNOME or KDE. Nobody could afford to go modernize Tk also, it was easier to just get all the apps on one or two toolkits. The modernization work was being funded or volunteered-for with Linux in mind, nobody cared to do it for open source toolkits on Windows or Mac. Swing was probably the only cross-platform toolkit that had the same modernization work done. (Ignoring "wrapper" toolkits like wxWindows or SWT that have whatever modernization is found in the toolkit they wrap.)
The Tk text widget was a solid piece of work, I mean, it's an accomplishment to write code good enough that it makes sense to port it to another toolkit. The core btree data structure from Tk is in GTK to this day with extensions but no real major overhaul: http://git.gnome.org/browse/gtk+/tree/gtk/gtktextbtree.c I think Ousterhout may have written it.
It's an interesting data structure if you like that sort of thing. The older GTK text widget had various operations that were O(n), and so people kept writing O(n^2) or worse code accidentally; with the new one, one of the goals was to try to keep everything below O(n) so it was harder for developers to hose themselves. The core of this was the btree from Tk. One place we declared defeat was in very long lines; some stuff is still O(n) where n is the line length, and that's hard to fix without massive overhaul. The btree is about storing lines and related metadata so that things aren't O(n) in number of lines. There's also a useful trick in http://git.gnome.org/browse/gtk+/tree/gtk/gtktextiter.c where an incrementing "change stamp" is used to keep an iterator valid even though the iterator caches some pointers that can become invalid as the btree changes.
Sorry for the "used to walk both ways in the snow"-flavored post ;-)
This still haunts me today. IDLE, the IDE-like editor that comes with Python, struggles with long lines. Execute something like this:
print list(range(10000))
Was it the decision of one man who just liked Java better? Was it for more technical reasons (like the lack of a standard OO system)... etc?
I don't know Tcl well, but I still feel like a different present in which Tcl had arisen to be one of the most popular languages may have been better than the actual Java-dominated present.
The other thing that makes it good for large systems is that Java was designed from the ground up for running untrusted code. I know the python module that attempted to offer the same functionality has been abandoned as basically unworkable. (Though interestingly this is one of the things that Tcl did well - safe Tcl actually worked, afaik).
That said, I haven't used it for a new project in over 10 years. I still kind of miss it though...
I used Tcl/Tk heavily in my 3D graphics class back in 1998, and while I enjoyed its light weight and ease of use, I felt that the GUI output was definitely old-school compared to Gtk and Qt, and had no desire to continue using Tcl/Tk after the class ended. It felt like the past.
My own tl;dr for this comment: David makes good points and his experience mirrors some of my own. I think the biggest mistake was not going with a "batteries included" strategy for Tcl releases.
1) Tk got left behind by Qt, GNOME, etc.
I whole-heartedly agree with this. We spent a very long time making our Tk apps fit in on user's Linux based desk tops - tweaking color schemes, widget dimensions, and even adding new widgets that Qt and and GNOME had that Tk didn't (trees, tables, buttons that could show images and labels, for example). Tk has since made this a lot easier but at the time it was frustratingly time consuming.
The author's other point that there were too many ugly Tk apps. Maybe. You definitely could spot a Tk app a mile away. But there are plenty of ugly X apps where X is any GUI framework so I don't buy in to that theory as much.
2) The debate between making Tcl batteries included or not.
In think this is the #1 issue. I love Tcl for quickly stitching together a few apps or processing files. Tcl makes working with the system really easy and it's "everything can be a string" roots can be tremendously useful. But... if you want to do something that's not in the core language - you had to build it yourself. There were quite a few useful external packages but they were often incompatible or tied to specific versions of Tcl etc, etc. Comparing that to the first time I built something in python and needed to process a gzipped file. Just import zlib. Wow!
Every big Tcl project I'm aware of had to devote a lot of energy adding support for some fairly basic - in comparison to other scripting languages - features. Tcl's philosophy was very much like the C language. Keep the core small - let the users work out the rest.
3. Missing default support for object orient programming.
Sure. Maybe. That kind of goes with point 2 above. It wasn't ever so much of an issue for me or any of the projects I worked on.
4. Missing support for some modern conveniences (no PNG, no readline based shell)
Sure. It goes with the batteries included or not theme. It was (and still is in some cases) a sign that tcl was falling behind.
5.Everything must be representable as a string
I spent a lot of time dealing with this. It's incredibly convenient when writing Tcl scripts. It's incredibly difficult when writing C extensions. The Tcl_Obj structure is actually really clever but managing the life cycle of objects is frustratingly complicated.
This is advanced enough of a feature that I don't think it impacts that many people. That said, fixing this flaw might have allowed for more interesting extensions.
6. (I'll add one of my own.) Multi-threaded tcl is not built in. You have to make the choice at compile time. Compiling in support for threads slows down tcl enough to be noticeable for the kinds of people that are interested enough to do multi-threaded programming with Tcl. :-)
Can you elaborate on that? I know nothing about that industry but I still have some nostalgia for Tcl despite of using Ruby now.
In the past I was a solid Tcl user (couple years ago even did a small custom Linux distro where bash was replaced with Tcl & installater was written in Tcl/Tk. It was fun!)
My thoughts on its ~ 0 popularity are:
* No Rails equivalent.
* No gems equivalent. In modern world it is a disaster. (One could notice how quickly nodejs got its npm.)
* 8.6 and OO system are in aplha/beta stage for 6 (?) years. Many of us just gave up waiting.
I still remember '2007 would be a great year for Tcl.' What a joke.
Cadence and Mentor Graphic's digital simulation GUIs were built with Tk. Probably a lot more. The big win for Tk in GUIs in the late 90's was it's cross platform support. For all of its limitations - Tk was a huge improvement over Motif!
Tcl is also used as a data file format for describing desing rules. Synopsys' SDC (standard design constraints) and Cadence's CPF (common power format) are both just design specific variations of Tcl. Writing a parser for these formats just requires writing command implementations. Tcl takes care of the rest.
Never realized that Tcl empire was so broad.
I also found a few bugs in the file selection dialog code. Someone on my team got an error during bug testing. I traced it to the file selection code built into TK and patched it there. Then I got curious, the same bugs were reproducible in ModelSim.
Lately I've been trying to embed Ruby the same way, and while I've succeeded, the process has been painful. The "event loop" is a particular challenge. So as far as I can tell Tcl is still leading in this regard. Can anyone chime in about Lua? I've heard some positive anecdotes.
On topic, it's probably easier to embed Lua than Tcl, though you may miss some of the libraries. The language is cleaner (in some sense). For an EDA application, the only real problem is what your customer is supposed to do with 20+ years of scripts that they've written that are specific to their flow :)
Some backstory: Thanks to Randy Pausch, in the early 90's I was fortunate enough to be exposed to Python and the Tkinter interface to the Tk widget set. With that experience I ended up building large and complex EDA apps for the SoC IP market in the late 90s. Python was the enabler because its language and library features supported programming in the large as opposed to Tcl with its arcane syntax (especially for those with a C background) and minimal library support.
I still pity the AEs who write Tcl scripts for EDA tools.
"even adding new widgets that Qt and and GNOME had that Tk didn't (trees, tables, buttons that could show images and labels, for example). Tk has since made this a lot easier but at the time it was frustratingly time consuming."
Can you open these up? More generally, is there a real CPAN for Tcl/Tk (and is there interest)?
The CPAN equivalent for Tcl has been tried a few times. See Teacup for the latest. None of these ever reached critical mass to gain wide spread adoption though.
If anything, I thought Tcl's scoping rules were too strict. E.g., global and namespace variables are not automatically available in functions, unless you import it explicitly.
[incr Tcl] was a great OO framework for writing GUIs. Perhaps not the greatest framework since it was trying to mimic C++.
As antirez said, "It's like learning Scheme or FORTH: mind changing."
Unfortunately, there aren't really any big killer apps written with tcl, and I haven't seen any major new books (a 2nd edition of Ousterhout's book and one or two others). I suppose it's better than the situation that Eiffel is in, though.
1) People hated pronouncing it "tickle", and would almost always initially call it T-C-L. This seems minor, but people have egos and if the name is silly, it's going to hurt in adoption.
2) Comments (#) are actually commands, and you can't simply comment out blocks of code.
3) There was no buzz or heat around the language, nothing exciting seemed to be happening with it. This led a lot of people to thinking it a waste of time to learn.
# if { $foo } { ...
I'm a bit surprised how language feature like this became the standard way of doing basic things. How is parsing "code strings" at runtime ever better than building an ast and handling variables as variables?
Expect is still pretty neat:
(which doesn't really change your point, but Tcl is directly involved there)
Languages need mindshare and to get mindshare you need marketing and tcl suck[ed|s] at marketing. Unlike python, ruby, lua.
The embedded vs batteries included example is a pretty good example of that: Tcl is neither small and limited/self-contained like Lua, nor does it ship with a bunch of stuff out of the box. So it satisfies no one.
I made the decision to focus on low level C++ / Windows right around the time tcl/aolserver was cutting edge.
I think my career advice for younger programmers is to stay away from whatever technologies I'm using.
I spent several years as a TK/TCL programmer. I believe where Tcl "went wrong" is that it never got attached to a very high profile use case or project.
It is not enough that the language was open source-- it needed to be used in a major open source product (or a major product period, the way flash is major, and thus ensured a life for lingo/actionscript, etc.) There was no Rails for Tcl like there was for Ruby. There was no Django like there was for python. There was no iPhone like there is for iOS, or massive number of open popular open source products like there is for Java.
I can barely remember anything of Tcl. It didn't make a big impression on me, even though I wrote it full time for a couple years at a job back in the day.
This is not true of other languages- Every language I've had to use full time has become my favorite language. At one point it was Java, then I discovered Objective-C and that became my favorite language, then I got into erlang, and now that is my favorite language (though it shares a spot with Obj-C because I use them for different things.) Tcl never made a big impression- it never gave me the big epiphany of a better way to do something that the other languages have (message passing in Objective-C makes it the best OO language in my opinion, and erlang's concurrency is unmatched, in my experience, though I don't claim to be an expert.)
Finally, the big claim to fame that Tcl had, was TK. Tcl programmers talk about TK a lot- notice how this article does. Unfortunately, the problem TK is trying to solve is essentially unsolvable in a good way. Sure, you can make a cross platform GUI toolkit, many have, but you then have to decide if you make your program look the same on each platform (and thus not like that platform) or if you try to make it look like the platform, in which case you run into all kinds of issues (because each GUI platform is based on seriously different design choices and thus having one bit of code support contradictory design choices on the different platforms is difficult. You can abstract a lot of it, but not all of it.) In the end, these cross platform apps never quite look right. Perfectly fine for Enterprise software (part of the reason Java found such success there) but never really delivering for the commercial shrink-wrapped market, which needed each platform app to look just right for that platform.
The two big lessons I take away from this for any open source project is- 1. Have a clear compelling feature that you absolutely nail. Be the go-to place for that feature (as TK tried to be.)
2. Do everything you can to make sure there's some high visibility adoptions of your project in other projects or products.
I read about half of the article before my annoyance surpassed my curiosity. Come ON!
This is constructive feedback.
Not really. Some stories, such as those involving hundreds of people over the years, many companies, countless person-hours of work, and lots of money, are long and complex with many details and cannot be summed up in one or two paragraphs. I mean, you could say about my article that the key point is "marketing is important", but you'd be missing out on many details.
Lord help us if you're ever confronted with something like...shudder... a book! Some contain thousands upon thousands of twisty paragraphs, not at all alike.
Or, as a mediocre man once said, in the future, "There was a time when reading wasn't just for fags. And neither was writing. People wrote books and movies. Movies with stories, that made you care about whose ass it was and why it was farting. And I believe that time can come again! "
The article in question would benefit greatly from a summary that allows the reader to decide if reading the whole thing is worth the effort. In scientific publishing an abstract is part of good form. In journalism the structures are usually less formal, but a good journalist will sell you a lengthly article without requiring you to read the whole thing.
I have been criticized for writing at length and being bad at summarizing myself. I know it is hard. And of course I find it annoying when people point out how bad I am at writing. But you know what: it is extremely constructive criticism.
I figured the title was enough to give away the general gist of the article: it's about a programming language and the problems it had. If that's a subject that interests you, perhaps the article is worth a read. I wrote the article to talk about a subject I know well, for people interested in that subject.
tl;dr: not everything can be dumbed down to TV-style sound bytes.
The article was not meant for people with a casual interest in the subject, but those who wanted to know about the details.
If that's not you, so be it. I would note, however, that at this point you've surely spent more time complaining here than it would have taken you to read the entire article.
If you're unwilling to finish it, that is okay. Coming here to complain about the length of a relatively short article doesn't add to HN, however; it just clutters up the discussion for those of us who read end enjoyed the entire thing.