Tcl the misunderstood (2006) | Hacker News Reader

Tcl the misunderstood (2006)

antirez.com

181 pointszeitg3ist108 comments

A little Tcl story. When John Ousterhout interviewed me to be VP of Engineering at the Tcl company (Scriptics) many years ago I told him I knew nothing about Tcl at all. He told me that the two great virtues of Tcl were that it was incredibly easy to learn and had very powerful regular expression facilities (I think he was trying to contrast with Perl which also has great regular expressions).

I blurted out "Don't those two things contradict each other? You can't have something that's easy and have regular expressions!".

I immediately figured I'd totally blown my chances and discovered afterwards from the recruiter that the one thing he really liked in candidates was to be challenged. Me and my big mouth got me that job.

Nice anecdote.

I'm curious what your thoughts on Tcl the language are today, given your former position at Scriptics.

My little Tcl story.

My first developer job was for a company which used Vignette StoryServer which used Tcl for its primary scripting language. You actually wrote all the Tcl inside a Java based tool (I think there was a way to use an external editor but it was clunky has hell(. We actually did most of our dynamic stuff in Perl via traditional CGI but all the straight CMS work was done inside StoryServer's tools. One of the vagaries of that system was that if you needed to escape an opening bracket "[" you needed to use 2x + 1 (memory is hazy on the exact number) the number of backslashes for every level of brackets you were in. So if you were 3 brackets deep you needed to escape a bracket it would be "\\\\\\\\\["

Otherwise actually I ended up kind of liking Tcl (in some ways I liked the language more then Perl, but the environment we were working in was pretty terrible) and that experience actually translated really well when I play around with objective C for iOS.

As others have said, most uses of Tcl was no doubt Eggdrop bots. I made an eggdrop bot to do some interesting stuff in an irc channel. I also learned nice and important lesson in the process about Tcl and dynamic execution in general (such as in SQL) : Fully dynamic execution is great as long as you spend more time checking input than actually writing logic.

As an example, you can do the following in Tcl as the article points out:

  set a pu
  set b ts
  $a$b "Hello World"

Awesome! I'll use this kind of dynamic behavior to let users of the bot have more freedom. Bzzzzt. This was back in the 90s and the idea of 'SQL injection' wasn't as widespread as it is now... Getting around these kind of injections in Tcl requires constant vigilance and sometimes is very confusing.

Another interesting issue is the lack of types and how Tcl interprets them. Ask the user for a number and check if its between 1 and 3, and reject if it isn't. Works fine until the user tries the number 0x01, which matches 1 in some places but not in others. Gave me a lot of appreciation for typed languages where an int really is an int.

You have to be similarly careful with shell scripts.

  $ a=ec
  $ b=ho
  $ $a$b "Hello World"

Sure, but people don't generally write large programs in shell scripts.

If you want to relegate TCL to short scripts, i guess it's the same as shell scripting, but if you want to use it for more than that this behavior is dangerous.

The phrase "SQL Injection" was coined in 2000. There was just about no understanding of the threat of these attacks in the 90s

I'm sorta confused when you say there was "no understanding". It's not like a basic SQL injection is something hidden, or even remotely difficult to understand, and it doesn't even exploit a complicated environment.

You're literally passing user input to a language interpreter - what else could you expect? I don't believe there needs to be "understanding" to realise that you just wrote exec($userinput) and that might be bad.

I'm curious if you were actually developing software in the 90s. Of course, in hind-sight, SQL injections are obvious. However, at the time, the collective understanding of security issues was much lower. Most people developing code were doing so for desktop users, where people would be "injecting" themselves, so no thought was given to the need to sanitize.

Those attitudes and behaviors spilled over into web development. I know I built more than one perl cgi script that concatenated user-input values together without thought or concern.

It wasn't until very near 2000 that the collective consciousness started to come around the new and different problems web programming brought. It still isn't completely there.

I believe that for many programmers who knew what they was doing in 90's the problem that we now call "SQL Injection" was about functionality ("Why can't my article contain an apostrophe?") and not security. It's similar to many buffer overflow vulnerabilities and so on: often it is not only security issue but also functionality issue.

Yes, but my first real commercial stuff was on the web. Web apps aren't that special though: it holds true for any server-based app. A point-of-sale app that allows a clerk to inject and change prices is pretty messed up. Same principles.

Although, maybe it's just a different mindset. Some developers seem to aim to develop "easiest thing that can work" and others aim for "easiest thing that won't break".

I think it's more of an Eternal September (http://en.wikipedia.org/wiki/Eternal_September) event where security was fairly well understood by a small group of people that needed it in the 80's, but 'those in the know' (Banks, NSA, etc) never really spoke with the new generation of web developers.

PS: There is a reason that phrase refers to September 1993.

I remember when Eternal September began. I wasn't exactly a seasoned veteren at that point, but I had at least enough history to have compiled a Usenet FAQ (I came online in 1990). It was sad what the Eternal September did to misc.kids. I still miss that group from before that time.

I was developing CGI programs in the 1990s. Some of my code was in cgi-lib.pl, which was the first widely popular CGI library. I can say that people knew about the problems of trusting arbitrary user input.

Lincoln Stein, who wrote the CGI.pm (the second widely popular CGI library), also wrote "Web Security: A Step-by-Step Reference Guide", published in 1998, which was two years after he started the "World Wide Web Security FAQ". Here's the 1998 archive of the FAQ: http://web.archive.org/web/19980213181713/http://www13.w3.or...

SQL wasn't that widely used in the early- to mid-1990s. While some places used it, I don't think they went mainstream until Philip Greenspun's "Database Backed Web Sites: the thinking person's guide to web publishing" in 1997. So early SQL injection examples are hard to point out.

But shell injection examples are a dime a dozen. Here's a posting from 1995 outlining the strategy for dealing with CGI input: http://groups.google.com/group/comp.infosystems.www.provider...

> The entire philosophy can be summed up as "Never trust input data." Most security holes are exploited by sending data to the script that the author of the script did not anticipate.

along with examples with the system() command:

    Perl:
        system("/usr/lib/sendmail -t $foo_address < $input_file");

    C: 
        sprintf(buffer, "/usr/lib/sendmail -t %s < %s", foo_address, input_file);
        system(buffer);

    C++:
        system("/usr/lib/sendmail -t " + FooAddress + " < " + InputFile);

plus variations through Perl's pipe, backtick, exec, and eval commands, and C's popen().

It even mentions that using a

    open(MAIL, "/usr/lib/sendmail -t")
    print MAIL "To: $recipient\n";

doesn't go through the shell but does pass data unchecked to sendmail, so the one should make sure the code cannot be tricked.

If you did HTTP in the early 1990s then you very likely used NCSA's httpd. The security page for httpd, from 1998, is still available from archive.org: http://hoohoo.ncsa.uiuc.edu/cgi/security.html .

> A well-behaved client will escape any characters which have special meaning to the Bourne shell in a query string and thus avoid problems with your script misinterpreting the characters. A mischevious client may use special characters to confuse your script and gain unauthorized access.

Replace "Bourne shell" with "SQL" and you have a SQL injection attack.

You wrote: "However, at the time, the collective understanding of security issues was much lower."

I think you underestimate the collective understanding of 15 years ago and overestimate the collective understanding of now. I've seen a bunch of code written in the last few years (mostly for internal use, by non-professional programmers) which still which sends arbitrary user commands to system() and to SQL.

SQL injection style security problems are merely a subclass of sanitizing user inputs.

Bad user inputs can cause crashes/catastrophic problems e.g. buying -1 shares from a stock market system.

Therefore computer science teaches to always sanitize inputs. I was taught this in high school in the 80s.

"I know I built more than one perl cgi script that concatenated user-input values together without thought or concern."

You have no excuse.

As someone who spent his teenage years "hacking" in multiple senses of the word, I can assure you there were plenty of people using buffer overflows before rtm's worm and there were plenty of people using data injection attacks long before the phrase "SQL Injection" was coined.

Back before the mid-2000s it used to take years/decades for ideas like that to bubble up into being "common wisdom" but in those years/decades, there would be hundreds to thousands of people exploiting the ideas, sharing information with other like-minded individuals privately (pre-everyone-has-a-blog), etc.

Wait, what? There were tons of people exploiting buffer overflows to upload code into running processes before 1988?

I call shenanigans.

Like you, I'm personally acquainted with a pretty good cross section of the best known people in vuln research in the '90s, and I was in the room with Peter Zatko and Dave Goldsmith and, at other times, San Mehat, Tim Newsham, and Ivan Arce as they figured out various ways to exploit overflows. This stuff was (weirdly!) new when 8lgm published it in '95. I sincerely doubt that it was old news to anyone when RTM used it in the worm.

If this was in any sense old hat to anyone, where are all the overflow exploits between 1988 and the Lopatic NCSA HTTPD exploit from 1995?

In your circles they may have been "just about no understanding", but in my circles there was.

For the record, the standard database interface that Perl uses, DBI, was first released as early alpha on Oct 12, 1994. Its solution to SQL injection is bind parameters. As far as I can tell, that first release had support for bind parameters, but I don't have that version's documentation for that feature.

The first release of the first real database driver was 24th Feb 1995. (Again, early alpha, you couldn't actually fetch data.) From the comments in the change log, bind variables were always a priority, and finally arrived for DBD::Oracle on 22 Aug 1995. From that point on if you used Perl and connected using DBI in the way recommended by the documentation, you were protected from SQL injection attacks.

I remember conceiving of the idea of Sql injection all by my lonesome in 1996. When mentioned this thread to my employers, I was old that "we do not to consider such possibilities since they exist in every part of our code".

So there was "no understanding" in only a particular sense of the terms... (I doubt I was the only smart person to conceive of these attacks back in the day)

I would believe you if you said buffer overruns. There was an understanding about buffer overruns, but the tools to help protect against them weren't developed until much later.

I actually was forced to use Tcl recently on a project, and I came across this article at the time.

It so happens that I was getting into Lisp at the same time, and the parallels between them are obvious. You effectively get macros, so it's a pretty expressive language. There's even an object system in Tcl based on CLOS. But ultimately, it feels like a language with lots of good ideas, implemented poorly. I'm not a PL theorist or purist, but "everything is a string" constantly feels like a really poor abstraction.

Tcl is practical for small, quick projects, but you soon run into insane things like the fact that curly brackets are syntactically significant in strings - and comments.

> curly brackets are syntactically significant in strings - and comments.

The latter trips me up far more than I would like to admit :(

It's a great example of the nature of Tcl as a language, though. Sometimes I love the simplicity, sometimes consequences of it drive me nuts.

For those who haven't used Tcl: You write comments like this:

    # this is a comment

But, in keeping with Tcl's simple and tiny nature, this isn't a special case. "#" is simply a procedure that ignores all arguments supplied to it!

While this is extremely elegant, and means you could implement comments in Tcl itself rather than the interpreter, it does mean that you have to balance your curly brackets within comments, which is hugely annoying when you forget.

I use Tcl for a number of analysis scripts inside some scientific software. Recently, I came across a situation in which I had nested loops:

    for {set i 0} {$i<$n} {incr i} {
        # Some comment
        for {set j 0} {$j<$n} {incr j} {

The script refused to run and eventually I figured out that the comment was somehow impeding execution. I still don't understand why.

This also breaks, because comments are syntax.

   proc some_proc {a b} {
   
   } # some_proc

That's easier to explain though - you are passing the arguments {a b} { } # and some_proc to the procedure some_proc. Because comments are just another command, the # has to be the first non-whitespace character on the line.

Based on the instructions in the article (tho I haven't tried it out), putting a semicolon before the hash would work...

    proc some_proc {a b} {
    
    } ;# some_proc

Are you sure the comment didn't have some curly brackets in it or something like that?

To my best knowledge, the hash mark comments are parsed as comments by the Tcl parser. The interpreter will not execute a "hash mark" procedure. You can check for the parsing and semantics of Tcl by googling for "Tcl dodekalogue".

I actually wrote some code to find unbalanced braces in the codebase I was maintaining at work. One big downside of Tcl is that the parser gives you practically no help with syntax errors--never a line number, and only sometimes does it give you context.

Another common mistake is that curly brackets are VERY significant in [expr]. The author writes statements like "expr $i * $i" in several places, and it makes me cringe for two reasons:

1- This code prints "5" because expr does a double-substitution:

  set a 5
  set b {$a}
  puts [expr $b*1+0]

What's going on here is that $b is evaluated (to the string "$a") before calling expr, then expr itself evaluates its argument:

   $a*1+0

to 5. This is a problem because $b can be anything--for example, a call to [eval] arbitrary code.

2- Calling expr without putting braces around the argument is unfathomably slow. Tcl precompiles procedures as much as possible, and calls to functions requiring dynamic evaluation (like eval or expr without braces) can't be precompiled.

I once solved a huge performance problem by finding and fixing calls to expr made without braces. What was a process that took 40+ hours to finish was reduced to around 45 minutes by this change alone.

edit: formatting

Why do I see people talking about this kind of thing in Tcl being perfectly okay, while PHP is nowhere near as randomly dynamic and gets a lot of hate?

I suspect for two reasons - first, that Tcl is always dynamic the same way and in clearly documented places, whereas PHP is a lot less consistent about it; and second, that in Tcl a lot of the core language constructs do so, so it's something you get familiarised with early rather than bitten by later.

Plus, PHP users seem to be most likely to either go "err, so?" or "YOU DON'T UNDERSTAND THE TRUE GLORY OF PHP!!!" in response to being asked about it, whereas Tcl users generally tend to respond "yeah, you need to learn about that and get used to it" - which tends to significantly alter the next comment made.

(not trying to imply that PHP doesn't have users in the third category, just that the first two are the more visible and it alters the discussion's tone)

Probably because Tcl has other redeeming qualities.

Command line programs are now passe in all except a few niches like routers, chip design etc, but IMHO there still isn't a better "/bin/sh++" that you can embed as command line shell interface between your C functions and the user.

Swig + Tcl + C/C++ works like a dream!

Lua?

I would make a distinction between a CLI (to be used by ordinary users of the program) and a scripting interface (to be used to "extend" the program, to be used by power users and/or other programmers).

And I would further submit that the same language may not be the right choice for both tasks.

Tcl leans further towards being a pleasant CLI interface language, while AFAIK Lua leans further towards being a pleasant plugin development language.

IMHO.

tclsh is a REPL (Read/Eval/Print Loop) coded against libtcl. libtcl is _easily_ integrated into any C (C++) development project. tclsh is but one.

I am working on a Tcl parser implemented in Python as a side project. Feel free to use it on your code base:

https://bitbucket.org/ul5255/code-snippets/src/61958ee922cee...

> But ultimately, it feels like a language with lots of good ideas, implemented poorly. I'm not a PL theorist or purist, but "everything is a string" constantly feels like a really poor abstraction.

I see where you're coming from, but IMHO:

Tcl's ideas make total sense if you understand why the language was invented. The name is the hint: Tool Command Language. It was never intended to be a "proper" programming language, AFAIK.

That is, a "sh++" CLI to be embedded in big programs. The role of the CLI is to be a thin layer between the data structures in the embedding program and the user.

The EIAS approach makes it easy to write commandline scripts. I've seen non-programmers abuse the hell out Tcl but the underlying C function still got the correct arguments.

You can also write your own object system in like half a day, I think you peeked, but didn't completely open the door.

We use TCL for a couple of core scripts at work because most of the tools we spend millions of dollars on in licensing use TCL as their embedded interpreter (I work in EDA/semiconductors).

I find the most unproductive use of my time at work is when writing TCL code. Switching to TDD has helped a lot, but I still find the language maddeningly frustrating.

I hate that this is both valid syntax with wildly different meanings:

   set env(VAR) "value"
   set $env(VAR) "value"

I hate that TCL error messages have no prefix, so there is no way to grep for them (the 3rd party tools merge STDOUT and STDERR to a single log).

I hate that the TCL error message line numbers are RELATIVE TO THE PROCEDURE instead of relative to the file.

I hate that the language is fully interpreted and not compiled, so you can have a syntax error in a code branch that isn't triggered just lying there for months if you don't fully unit test every code path.

I do not understand how anyone could like TCL unless they were using a linting tool to give the robustness of debugging that most other languages have.

Because of our code base and the 3rd party tools I'm not able to lint our code and it is a brutally painful language to debug.

re: TDD -- tcltest ([package require tcltest], man n tcltest) is downright brilliant. Brilliant. It's worth noting that Tcl is the core of the GCC testsuite (dejagnu) as well http://www.gnu.org/software/dejagnu/

I also created a very rspec like bdd library, called tclspec: https://github.com/arthurschreiber/tclspec

Just out of curiosity (having used Tcl shell EDA tools myself) could you Swig stub versions of the proprietary commands and test your scripts standalone?

If you haven't already, you might want to look into Nagelfar. http://nagelfar.berlios.de/

One of the biggest advantages of TCL in my opinion is the simplicity of its syntax. The lexer is quite simple, straightforward and easy to create. Properly written TCL interpreters are quite small and memory efficient. Because of this TCL used to be quite a popular choice as an embedded scripting language for various appliances (eg. Cisco used TCL for their VoIP/IVR scripts).

Nowadays Lua has mostly taken TCL's place as an embedded scripting language of choice and I must say I'm a little sorry, there are some things that I miss from TCL.

Actually, I think its syntax isn't just simple, it's simplistic. It's simple to implement, but it presents some rather unexpected behaviour to the user. To wit:

  proc hello {} {
    # I'd like to put an unmatched { in my comment
    puts "Hello!"
  }

This doesn't compile because of the unmatched {.

But this does, by adding another unmatched } - in the following string:

  proc hello {} {
    # I'd like to put an unmatched { in my comment
    puts "Hello!}"
  }

In other news, Richard P. Gabriel can stop arguing with himself about whether worse is better. In this case, worse is pretty silly!

This is one of the reasons I'm only a little sorry that Lua has taken over. Not without its shortcomings, Lua is a big advancement in this regard! :)

Why didn't you just escape the brace with "\"?

That's not the point.

The question is not "How can I put an unmatched curly bracket in my comment?"

The question is "Why are the semantics of the language - including, but not limited to brace matching - so peculiar and unexpected?"

Look at my second code example! The matching brace is in a string, in another command, on the next tine! And it works! And removing either one breaks the code!

Why would you escape something in a comment??

I will be borrowing this: "its syntax isn't just simple, it's simplistic"

I've lost hours to the syntax errors in comments before. Coupled with TCL's bad error messages it's maddening.

   #syntax errors in comments are dumb
   
   proc test {a b} {
   } # test


   % source tmp.tcl
   wrong # args: should be "proc name args body"

How long does it take to find that one in a 1000 line file? It doesn't even give a file name or line number.

You might try Nagelfar, an extensible syntax checker: http://nagelfar.berlios.de/

Nagelfar can easily find these types of errors, plus you should be able to define syntax for your own commands (as in your previous comment.)

tclsh nagelfar.tcl engtech.tcl Checking file engtech.tcl Line 1: E Wrong number of arguments (5) to "proc" Argument 4 at line 2 Argument 5 at line 2

Comments are warty in Tcl, you would be better off to understand Tcl's syntax conventions, rather than assuming comments work like other languages. See item #10 at: http://wiki.tcl.tk/10259

Yes, Cisco IOS had TCL embedded in a few places. You could run scripts from the command line and access SNMP MIBs, run scripts on certain conditions, and I'm sure there's more.

F5 load balancers also use TCL for their iRules. With such a simple language you can do almost anything, such as rewriting the request, split test, fend of DDOS attacks.

TCL has quite a history with the network community. The "expect" language is built on top of TCL and was a popular way of automating CLI tasks on routers/switches before better tools came around.

TCL is still used extensively in the EDA (Electronic Design Automation, i.e. hardware design) industry.

As someone who works in the EDA industry, I can't say that our adoption of a given technology is exactly an endorsement!

Compared to peculiarities of user interface and general buggyness of most EDA packages I've seen all gotchas in Tcl's syntax are pretty insignificant.

Oh man, my only experience with TCL was a short-lived attempt at using it to do some stuff on a Cisco AS5400 (TDM/VoIP gateway) in 2005. It was terrible.

I found out that with certain bugs in the TCL script, the device would hard crash (dump to console and restart). On top of that, Cisco's documentation was flat-out wrong sometimes, with incorrect names for functions. Even getting it to timeout a phone call after n seconds wasn't workable; at best you'd get 10-15 seconds resolution.

There's no shame in calling Tcl a toy. It provides a fun way to learn lispy concepts and it has helped teach many that a simple lanaguage can be powerful. But I think Salvatore was unwise even in 2006 when he said Tcl isn't a toy. He is clearly a hacker in the best sense and his writing's appreciation for Tcl shows his respect for its power and simplicity. He was not the first, nor will he be the last, to sing such praises.

Sadly his apparent ignorance or lack of concern regarding Tcl's numerous flaws leads me to question his engineering maturity. For instance, nowhere did he warn of how simple typo errors in Tcl's variable names and interpolated strings leads to a layer of engineering hell beyond anything Dante imagined, a place I've been and vowed never to return.

If you really think you need to expose something like Tcl in your product, please think again. These days you would be better off with something like Javascript or Lua. I strongly suspect the Salvatore of 2012 would agree.

>Sadly his apparent ignorance or lack of concern regarding Tcl's numerous flaws leads me to question his engineering maturity.

</comic book guy>

Two of the most powerful languages I know are also very simple. Lisp and Lua come to mind. While they can't compete with Python or Ruby in therms of libraries, their syntax is far simpler and more powerful than any other language I know. I wonder why they are also much less popular, especially given that both are relatively fast.

The author missed one of the other huge advantages of Tcl: it is extremely easy to add Tcl to a C++ project as a scripting language. It took me about half an hour, and I had never done it before. Once added, it makes interactive debugging of big projects almost pleasant.

This is why I love Tcl too. As a general purpose language I'd rather use something else, but for projects where you want to make it easy to redefine certain behaviours without rebuilding and you don't want to implement your own DSL (and you shouldn't want that, trust me...) Tcl is right there waiting for you.

I find myself building projects where I implement system specific stuff in C with Tcl bindings and then use Tcl to tie the libraries together. It makes for modular software and easy testing, both unit testing through tcltest and testing stuff interactively in development.

It's not The Best Language(tm), but it really ties the room together.

> It's not The Best Language(tm), but it really ties the room together.

Haha, very well put. A lot of the time, I just want something that lets me tinker interactively with C/++ code (changing variables, calling methods, etc) to make sure it's doing what I want without having to recompile, dive into the debugger, or write a custom command interface. Plus, some things are just far less painful to write in Tcl--namely, anything to do with strings.

I've been pretty pleased with Tcl for this as well. Its syntax is convenient for interactive use and fine for actual coding in, without being faintly insane like many shells.

The main drawback to it I've found is its insistence on being able to convert everything from an object to a string and then back again. It's all too easy, when using certain functions (I think `lappend' is the one that caused me a lot of bother), to have your objects silently converted into strings. This is one piece of shell insanity that Tcl has kept. It's not a good one.

If your objects aren't really representable as strings - in my case, I'm storing a pointer to some C/C++ object in a TclObj, and relying on Tcl's refcounting to manage the lifetimes - this is what you'd do in something like Lua - then this can leave you a bit stuck. And even if you did support this, this could well leave you with two equivalent objects where you'd rather have one individual one. Tcl should take a leaf out of Lua's book, or Python's, on this matter.

A shame. It's not the only thing I didn't really love, but it was the only thing that really bothered me. I considered changing language, but the Tcl syntax is very convenient. So in the end I just worked around it by reimplementing a bunch of list primitives that didn't turn things into strings so damn often, and used those instead.

Agreed. List functions stringifying everything in sight is really inefficient and error-prone, and definitely one of the things that I dislike about Tcl.

For most purposes I've used it, the convenience of the language outweighs its problems, but I've thought about rewriting some of Tcl to fix the minor annoyances like these.

For me, the best feature of Tcl is actually not part of the language itself. It's a tool built on top of Tcl -- Expect.

Expect can be used to automate CLI tasks, but for me it is indispensable for testing programs with text-only UIs (I get to write a lot of those).

http://www.nist.gov/el/msid/expect.cfm

Ah, TCL, I knew thee well.

Back in the 8.3 days, when expect was still a commercial add-on form ActiveState for TCL on Windows, I implemented my own Expect command in TCL based on 'socket'.

If you ever find yourself using Expect with telnet, if you need more performance, you can implement your own Expect using socket which will run at least 10x faster.

The only trick is sending the right TELNET opcodes to get your target device to actually respond with a prompt!

there's a python replacement: http://pexpect.sourceforge.net/pexpect.html

You can also find Expect ports in other languages. For e.g.:

* Perl - https://metacpan.org/release/Expect

* Ruby - http://rubygems.org/gems/expectr

If any of you wonder whether you use an application everyday that still utilizes Tcl/Tk, you may be surprised (https://github.com/git/git/tree/master/git-gui/lib).

And holy crap it's ugly. I have a locally modified copy that uses ttk and the modern theme just so I don't have too look at motif widgets all day.

So, here's this article purporting to answer the misconception that Python is superior to Tcl in every conceivable way and, erm, I left with the impression that Python is superior to Tcl in every conceivable way except that Tcl is a smaller language. If you really want to do meta-programming in Python by putting together strings then eval() is right there waiting for you... but there are good reasons people try not to overuse it.

Yes, I believe you are missing the point. For example, when it comes to syntax Tcl is like a shell language in that you need very little punctuation to do stuff if you are just calling functions with string arguments. Python, on the other hand, forces you to fill everything with parenthesis and explicit quotes.

And while this kind of syntax issue might be seen as superficially trivial, consider the popularity of templating languages for generating HTML and other textual content. After a certain point its simpler to use a language where text is the default and code is marked up then it is to use an equivalently powerful library where the code is as usual but all the strings need to be quoted.

I've used tcl for years and I use python for my own projects.

The key difference is the lisp-like expressivity of tcl vs the structured nature of python, which makes it harder to do things like override control structures or rewrite functions on the fly.

Everything in tcl is a string leads to some very interesting consequences, some of which are outlined in the article. It's also easier to write disastrously messy code in tcl than in Python, and the freedom leads to rival and poorly documented libraries - it's not an "institutional" language the way Python is.

Syntax is a red herring.

I use TCL in a consumer app that I develop, to allow users to load and run scripts that control the behaviour of the app. Its for controlling cameras (smart shooter), and actually I usually end up writing the scripts for my customers. But its amazing what you can achieve quickly and easily due to how flexible TCL is, users are amazing when I email back a custom script for some fancy HDR photo bracket the same day they request it!

And I myself am surprised at how many photographers have managed to write their own scripts, even non-programmers. I've been thinking about adding support for python as the scripting language, but I can't justify it just for been fashionable.

My first experience with Tcl was before I knew anything about programming. Tcl was the scripting language for Eggdrop IRC bots, which back in the day was pretty much the only IRC bot around: http://en.wikipedia.org/wiki/Eggdrop.

I remember being annoyed by Tcl back then when I was modifying existing scripts. Although now I can see that it probably isn't the easiest language to learn for a beginner.

I'm sure a lot of people's first, and probably only, experience with Tcl was from Eggdrop bots.

It's pretty damn easy to learn. I think what you mean is that for someone who comes from C-like languages, there's a lot you have to unlearn.

I disagree. The really poor feedback from the interpreter on errors makes it incredibly difficult to form an internal model of the language.

"Well, maybe it works this way.... no. damn. Well, maybe this way... no. damn."

By the time you get it right, you've gone through so many possible models in your head, it's just mush.

I had to use Tcl/Tk for a Uni project way back when, at the time I wasn't a fan but I looked at it again slightly later and appreciated how powerful it could be when you scratched the surface. It's definitely misunderstood, but if people give it a chance there's a lot to like.

My first real programming job involved building web apps using OpenACS - http://openacs.org/. OpenACS was all Tcl executing in AOLServer and used postgres or oracle. It was awesome.

I Learned a lot from "Tcl for Web Nerds" and its companion "SQL for web nerds" (all links listed here: http://openacs.org/doc/ ). Back when I started Tcl, I was more like a emacs-lisp script kiddie. Because Tcl was so easy to learn and because OpenACS was such a great MVC style framework, it didn't take long to master the framework itself and dive into the interesting things that were related more with the architecture of a web-app and not just the syntax of a programming language.

The "whereas" poem, as I like to call it, that announced Miguel Sofer's inclusion in the Tcl Core team was real fun to read too - http://code.activestate.com/lists/tcl-core/1983/

Miguel Sofer had described an algorithm for representing hierarchical data in a RDBMS. What he put forward could be thought of as a different kind of a nested set representation. Whereas the nested-set approach involved keeping track of two numeric values (left & right) for each node in our relational records based tree, Miguel Sofer's algorithm would use the ability to lexically sort a base159 encoded string. This way, tree operations could be implemented via sub-string matching and sorting.

His algorithm was implemented in OpenACS because it allowed an efficient implementation of the OpenACS nodes table. Each URI in OpenACS has a node record associated with it and all these nodes are hierarchical records - http://openacs.org/forums/message-view?message_id=16799

The OpenACS nodes system allows the implementation of a fine grained permissions system which enables a child node to auto-magically inherit the permissions of its parent node - this is, if the child node didn't have any specific permissions set on itself - http://openacs.org/doc/permissions-tediously-explained.html

When I was learning ruby I implemented this algorithm using active-record - https://github.com/gautamc/hierarchical_objects The utdt.edu link hosting the pdf that Miguel Sofer created is broken now; I found a copy here: http://www.tetilab.com/roberto/pgsql/postgres-trees.pdf

I remember OpenACS (or even its precursors). Still running some sites out there, like photo.net.

In the late 90's I joined a startup that had its application server similar to OpenACS (Apache + in house dev Tcl module + DB bindings).

With a way of doing objects to table mapping, similar to what Rails has.

This back in 1999.

Actually this dates at least 97-98 :-)

Ah those were the days.

Yeah, thanks for correcting me.

I joined a bit later, remember? ;)

Thanks for that tree algorithm. I wish I'd known about it a couple years ago.

Some of my best programming stories are from hacking TCL on Aolserver. The ability to make up language constructs was so powerful, and something I haven't seen anywhere else - full disclosure, I haven't used lisp yet, but it looks uglier to me than TCL and a clean programmer can produce very nice looking TCL code.

Among other things I remember adding named parameters to tcl procs in like a few hours - a feature some other languages consider a plus.

Is it missing in TCL? Just add it yourself :)

For a while TCL was also the state of the art in web applications development:

http://en.wikipedia.org/wiki/AOLserver

Around the same time there was CNET's tool which became a commercial CMS http://en.wikipedia.org/wiki/Vignette_StoryServer

StoryServer was interesting, its one of those things that in at least one way was ahead of its time. It actually generated static HTML files instead of doing straight pulls from a db. It's "dynamic" nature was actually basically a 404 handler where if you hit the URL for a story and there wasn't a cached html doc it would then generate one from the db, write it to the filesystem and then output the page.

With all the discussion about the syntax, some of Tcl greatest features are overlooked. The article could do with an update. I would like to see the current state of Tcl compared to node.js Possibly the tcl core beats node.js on many points.

A few years back I researched for a scripting language for an embedded project. The choice came down to Tcl, Lisp and Lua. We went with Lua and liked it a lot.

After reading this article I wonder how Lua and Tcl compare. It seems like Tcl is more like Lisp than Lua really. Does anyone have experience with any two of those languages and would care to comment on their differences and similarities?

My exposure to TCL, or rather dialects were through Autodesk Maya (The MEL Script there seems a bit like TCL, IMHO). The other thing was Metal Gear Solid (PSX). Konami used TCL to set up the level entities - enemies, traps, etc. Internally each command was hooked through a "C" function that took argc, argv, and to my shock (and then grin), there were tons of "C" functions that we working like main() - e.g. parsing arguments. For a moment I thought - this must be slow, then I understood - it did not matter - it served it's job, and allow non "C" programmers (game designers, scripts) to set-up objects easy in the "C" world. The whole thing had to fit in 2MB along with textures.

Later working at our studio, I've stumbled on couple of sound tools written in TCL, and lately had to dabble once in a while in the MacPorts land (also TCL).

But then most of the people I've asked they don't know about it...

I love this article. The few times I've worked with Tcl I've found it an absolute joy to use. I just hope that people don't mistake his conclusion.

The core benefit to Tcl is also its main detriment. Tcl takes off in places like OpenOCD because its incredibly powerful simplicity makes it incredibly tiny. You can throw it in pretty much anywhere as an embedded language and it will buy you a TON of functionality. If you want it to be performant, however, you'll wind up writing a lot of piecewise optimizations which will ultimately cause you to lose the size benefit.

So no, it's not a toy language by any means. And yes, its simplicity yields incredible LISP-like power. However, it's not a language I'd _ever_ use to write the core of anything where performance is of concern.

Back in the day TCL was the most efficient way to do a GUI prototype in a non MS Windows environment.

Yes; one of the primary reasons I choose Tcl for certain personal projects is that it has a remarkably easy to use GUI library. The notion of "text variables" even introduces a notion of reactive programming.

Never used Tcl, nice overview.

  set a pu
  set b ts
  $a$b "Hello World"

Seems pretty cool. uplevel is neat as well. The modifications = DSL approach seems interesting, too.

Uplevel essentially allows macro like functionality to be implemented - although it is executed at runtime rather than compile time (you are executing in your caller's scope).

However, it seems slightly evil for general use unless really needed.

Playing with uplevel is really the first time I understood stack frames.

Shorter Tcl Complainers #1: Why does Tcl require me to learn how to use it instead of just letting me carry over my assumptions from other languages?

Shorter Tcl Complainers #2: Why does Tcl punish me for my insistence on following bad programming practice?

Shorter Tcl Complainers #3: I hate Tcl because of traits it shares with several popular languages which I love!

I usualy use Tcl for my personal projects. It's nice and simple but powerful language, and the only real problem with it is that there's too many old libraries that can't work with multiple but different versions of Tcl on the same system.

Same here. And having all the code from the ground up is very useful. TCL turtles all the way down!

I like Tcl. I like that it is quirky and that it different enough (non-algol) to be interesting to me. I like the folks in the community as well. BTW 8.6 is just around the corner.

I wrote a lot of Tcl back in my days and the one thing I remember is the awful scoping rules, e.g. if you want to reference a global variable, you need to say so first, otherwise you'll be referring to a non-existent local variable and you won't get any error, just an null value.

It was horrible.

Tk was nice, though.

Unmatched brackets in includes.

Bane. Of. My. Life.

Would it lessen the pain to know that this doesn't just afflict TCL? Say, if your vendor (who is charging you $30k in licensing) forgot to close an 'extern "C" {' in one of their system headers? Something that could have been caught by a simple "include all the headers (twice) and compile" unit test. And this in a supposed "real-time" OS.

My first job coding was in Tcl/Tk. it was a vendor system for trading,accounting and only supported Tcl/Tk even for APIs. It was fun. I had never heard of Tcl before that.

I think people who used Tk with other languages (which TBH it didn't really fit cleanly with) were scared away from Tcl.

I never found a use for Tcl myself but I had no problem with it.

TCL has the closest syntax to English in term of defining DSL. Other languages claiming DSL support is just glorified functions.

upvar was a unique feature of TCL that was also a really bad idea.

http://www2.tcl.tk/1508

From this, it looks like Redis inherited some of Tcl's syntax. I never would have put that together before reading this.

I first read this as TTL. which is also often misunderstood.

Here is SICP author Hal Abelson's treatment of Tcl from yesteryear: http://philip.greenspun.com/tcl/

There are more legacy Tcl articles at "reddit.com/r/tcl".

Reply on news.ycombinator.com