Tcl the misunderstood (2006) | Hacker News Reader

Tcl the misunderstood (2006)

antirez.com

131 pointsthrowaway34445 comments

An interesting corollary to concept 5 (the fact that everything in Tcl is a string) is that this means that Tcl is actually homoiconic. It does it completely differently from Lisp, but still ends up in the same place: code and data have the same representation.

This is one the reasons I like parsing HTTP requests through TCL. Everything in HTTP is a string, there are no numerals. Typecasting arguments from query strings is the source of many ills. It isn't much of a problem with TCL since like HTTP, everything is a string.

Everything else about web service backends in TCL is pretty sad and frustrating, though. Unless it's embedded, anyway.

Everything in TCL is a string. Until you treat it like a number. And it has a leading zero. Then it's octal.

(Understanding TCL lists are just delimited strings lets you do interesting things. Sometimes unintentionally.)

You can represent arbitrary C++ code as a string, too, and C++ has a fairly decent string type. Does that make C++ homoiconic? I wouldn't say so, because the structure of the code is lost when you represent it as a string and because C++ doesn't give you any way of using the code represented thus (e.g., there's no standard way to compile it).

(The second of those isn't strictly part of what's meant by homoiconicity. You could, after all, write your own compiler, just as you might write a metacircular evaluator in Lisp. But it seems like an important part of the spirit of the thing.)

Tcl may be homoiconic -- it's been years since I looked at Tcl and I don't remember anywhere near enough details to try to pronounce on this -- but if so, it isn't merely because it represents everything as strings. It'll be to do with simple string operations being able to do all the parsing you need (on account of the syntax being simple and/or there being string operations that do things like parsing out a {...} block of code), and with there being a usable eval.

Tcl has eval and uplevel, which allow you to easily execute arbitrary Tcl code within the language. You can easily implement macros, lambda expressions, list comprehensions, and more using these tools, if you want to.

TCL code in the form of a string is not a data structure the language provides you tools to manipulate past being able to execute it: if you wanted to write a macro you are forced to parse the string. While the code is conceptually a list of commands, each of which is a list of lists, with code strings as a special case of value that is itself really a list itself, those semantics are not what you type: code is a semi-colon separated list of lists (which could contain, of course, nested semicolons), and cannot thereby be parsed as any normal TCL data structure (nor can it trivially be run through split or something to get a normal TCL data structure). The square brackets are themselves somewhat special, and the white-space has complex semantics; variable substitution is a one-off feature. The language thereby has the property that code is data (so the example about C++ given by gjm11 was probably confusing, and even got me at first: I was about to write a response to him talking about how in C++ code is separate from data in a way that it isn't in TCL), but it isn't really "homoiconic" (which I then realized as I started working on that response, as it has been over a decade since I lived and breathed TCL, and all I had remembered was the abstract conceptual beauty as opposed to the nitty-gritty details of actually typing it; I started typing it, tried to manipulate the string, and failed). Due to this, I would quibble that you can easily implement macros: you actually first have to write a TCL parser (which people have done; if you search for tcl parser you find some TCL parsers written in TCL in the package index) to recover the structure of the code from the opaque data. Like, you may as well be coding in JavaScript: no one would claim JavaScript is homoiconic, but they do define a .toString() on functions which is supposed to return a string with code that could be evaluated back to a function... you technically can then write macros that take as an argument a function that is .toString()'d, run the code string through a JavaScript implementation of JavaScript's grammar (such as Narcissus), and then manipulate the result to return replacement code in the form of a new function... if you've ever worked in an actually-homoiconic language (like Lisp) this is a fundamentally different experience.

Hi, downvoters! Would you care to tell me what you disliked about my comment, and thereby help me do better in future? (Or allow me to defend myself if I don't see what you disliked as a fault.)

I should maybe clarify one point: I was not saying that Tcl isn't homoiconic (which is, e.g., why I said "Tcl may be homoiconic"). Only that the mere fact of representing everything as a string doesn't make it homoiconic.

Interesting. I've never worked with Tcl, is it possible to write Common Lisp loop macro in Tcl?

I'm not familiar with lisp loop macros, but here is how this works.

For example the Tcl does not have try-catch-finally statement. If that happens in other languages you just need to accept it. In Tcl you can implement it yourself for example:

http://code.activestate.com/recipes/68396-try-catch-finally/

I like tcl, but in my experience it's no lisp.

You can do macro like things in tcl, but very often the evals and uplevels all worked to make my brain hurt in ways that writing cl macros never has.

It's not possible to write CL's loop macro per-se (tcl does not have macros, but commands/functions which are executed each time and there isn't an distinct macroexpansion/compilation phase), but it's certainly possible to write something that essentially matches behavior and user interface of loop (and it's probably significantly easier to do than writing loop in CL).

TCL is great at being an embedded language, especially when you want something that looks like a DSL using safe-interpreters. (Basically, these are sandboxed interpreters with access to limited built-ins, plus any methods you might inject into the global namespace)

TCL lends easily shell-like syntax so it is great for adapting your DSL to a REPL and even maps fairly cleanly to REST.

Years ago, I built a REST-like API that had one URI to which one could POST a TCL script. That script in its most basic form would be a series of pipelined API calls. There was even limited transactional support. I saw this important for mobile applications as it would reduce latency.

While the scripts were technically TCL, it was easier for developers than Javascript or Ruby (which we also beta'ed). It didn't look as much like a programming language as it did a series of shell commands.

The biggest problem with the above is that TCL safe interpreter still allow loops and other blocking operations. It means that you need to write a reaper to kill long-running threads / processes.

Combining such techniques with ZeroVM (or even Docker, or both) would be interesting.

Since Tcl 8.5 (Dec 2007) interps have timed resource limits available for them.

  bch$ tclsh8.5
  % interp limit {} time -seconds [expr {[clock seconds] + 15}]
  % while 1 {set a 9} ;# 15s elapse, then...
  time limit exceeded
  bch$

edit: Add release date for Tcl 8.5

I agree with the sentiment, but who is actually claiming that TCL is a toy language? Random uninformed people on Reddit opine about all sorts of stuff.

It is an old language with lots of cruft, but it's been used in all sorts of places for something like 25 years. Perhaps people's experiences with the language are colored by using it as a means to customize behavior in some big hairy enterprise app.

I don't love TCL, but "toy language" is frankly a comical representation. At least from what I see, it's practically the exact opposite. Nobody I know uses TCL because they like it or because it is fun; they use it because they must, to get work done (to control big hairy enterprise software).

> Nobody I know uses TCL because they like it or because it is fun

I haven't written TCL in a few years, but back when I did (when I worked on MacPorts) I actually did think it was fun.

Tcl is certainly not a toy language. I was introduced to it when running an eggdrop bot in the irc days, and I suspect for many of us was one of the first more feature complete languages we had no choice but to learn because we wanted to add custom functionality to our bots.

I haven't kept up with it and enjoyed seeing there's still those who find it productive

TiVo used to use it pretty heavily. I'm not sure if they still do.

Well, back in those days (around 1998-2002), when I was a kid at school, Eggdrop bots were a bit slow, which often became an important factor on channel wars: channel overtakes, net splits, and so on.

So we (some friends and me) created our own bot (with a very simple DSL) just for fun in pure C, and it was very fast (http://davis.sourceforge.net/). Of course, our DSL was never as well designed as Tcl.

Haha! I didn't get much into channel wars, diplomacy got a lot further when there was always legit bandwidth kicking around.

Tcl in some ways was the cause of, and solution to a lot of problems on IRC.. I ended up becoming friends with some of the people who'd try to takeover a channel I might be in and we'd realize we had more in common than not, such as how to customize those damn bots. Still have some of those friends years later, and I'm not sure if I'm the only one. Tcl feel good story.

I still have an eggdrop. Uptime nearly 200 days!

Respect! I feel like firing up mIRC but I don't have windows nor the patience to fire up my vm :P

Funny, that recently I was thinking how elegant and LISPy Tcl is.

I just finished building a CAD-like visualization utility in Tcl/Tk which using a complex Tcl-based internal DSL to define complex multilayer geometries.

Tk and it's Canvas widget are great for things like this.

Many years ago I also wrote an IDE for Image Processing algorithm development in Tcl/Tk, which also heavily used Canvas. I was able to build a complex GUI Imaging application literally in the wish shell, all while learning Tcl/Tk and [Incr Tcl] OOP system.

The trick was to bind key like F11 to reload source files and redraw GUI.

For the Image Processing stuff or anything else compute intensive - Tcl was very slow, so I either spawned external processes or used native implemented Tcl commands.

Tcl versions changed native command API, so many native extensions weren't ported to newer Tcl versions. I have some code that uses old 2001-based Tcl, because the native packages weren't ported.

I know that there is a package manager for Tcl, but I don't think it's widely used.

So, if you need to glue many different utilities with simple control logic and put some GUI above it - Tcl/Tk is great.

Back at the beginning of time I worked with Vignette StoryServer, which spun out of aolserver, one of the first "database-backed dynamic website" servers. The scripting language was TCL 7.something.

I never encountered it again, but its elegance always stayed with me. Except for uplevel. Fucking evil.

uplevel allows you to write control structures like if, while, etc... in Tcl itself. Yes, that's something you need to be careful with because it's extremely powerful.

It gives you the capability to extend the language itself, something you can't really do in most other languages. Tcl, Forth and Lisp are the only ones I can think of. These days you get "monkey patching" and so on, but it's nowhere near as nice.

Rebol, Smalltalk, Perl6 and Io are some other examples that have the capability to (easily & cleanly) extend the language itself.

re: uplevel - Interesting Perl5 has this feature via a CPAN module! - https://metacpan.org/pod/Sub::Uplevel Also another variation is https://metacpan.org/pod/Scope::Upper

This is me, as well. TCL seems wierd and clunky when you come to it from C-derivatives, but once you get used to it, it's possible to write some very elegant code.

Tcl might've been elegant but StoryServer most certainly wasn't. Http variables automatically injected into the code was a particularly fun security risk. Then they provided helpful (to hackers) error messages that could be googled. Still came up with a result all these years later! :-) http://www.google.com/#q=intitle%3A"StoryServer+Error"

> i18n just happens

No it doesn't... unless machine translation is "there" and built into Tcl.

> Every string is internally encoded in utf-8, all the string operations are Unicode-safe, including the regular expression engine. Basically, in Tcl programs, encodings are not a problem - they just work

Really? Perl's Unicode support is pretty much second-to-none, IME, but you still need to know the encoding of your file handles and so on. Once it knows this, Perl will Just Work(TM). How is this handled in Tcl?

I don't understand the purpose of your comment. OK so perl is great. This article isn't about Perl...it's about the semi-obscure and commonly underestimated Tcl (which is also a great language).

I don't think fuzzix was trying to divert the discussion onto Perl, but citing it as an example of a language generally agreed to do an excellent job of handling character encodings in which it still isn't true (as the article alleges for Tcl) that "encodings are not a problem - they just work".

The purpose of the comment seems clear enough to me: to point out two claims made by the article that seem implausibly optimistic:

1. That i18n "just happens", which as fuzzix says is surely impossible unless Tcl includes magical machine translation facilities.

2. That "encodings are not a problem - they just work", which would require Tcl to tell by some ingenious means what encoding is used by any given file -- something that Perl, despite being exactly the sort of language that doesn't mind the kind of heuristic complexity it would take to do this well, and being generally acknowledged to do a good job of handling Unicode, punts on).

To state the problem explicitly, internal storage format of strings doesn't mean you can handle all data streams seamlessly. Internal consistency is fine, but ultimately your program will have I/O.

If I open an ISO-8859-15 or UCS-2 file in a Tcl program, how is this handled? Does it seamlessly detect encoding and translate to UTF-8 or is there more to this issue than what's stated in the article? Also, are the strings stored in a canonicalised form? If not, how simple is it to perform normalisation?

I brought up Perl because it's my experience of a language with excellent encoding support and helps make the point that the issue is probably more complex than just how strings are stored and manipulated internally.

This article is from 2006.

Perl's unicode support is excellent now. In 2006 5.10 wasn't out yet, and anything before 5.8.5 had all sorts of odd unicode bugs.

Or: Tcl got there significantly before we did.

That is a fair point, but I'm not trying argue that Perl is somehow better. I am just using it as a reference point. I'm more suggesting the article may be overstating the ease-of-use / capabilities of Tcl.

To be clear, I'm not arguing against using Tcl in favour of Perl or anything else. It's just that my experience with a language with great encoding support doesn't jive with what's claimed here.

Well I think that particular clause was written for C-style language users where the unicode support is definitely subpar.

I came away from this impressed, but still thinking that it is a "toy language" in the sense that it is too powerful. You sometimes do not want to have to worry what a line of code means, and TCL does not allow that.

> The Tcl community developed a number of OOP systems, radical language modifications, macro systems, and many other interesting things, just writing Tcl programs

Well there's your problem. It becomes impossible to learn the language as a whole, because it is so flexible and everybody does things differently.

You call that a "problem". C++ developers make money with that every day.

>You sometimes do not want to have to worry what a line of code means, and TCL does not allow that.

Comments in Tcl work perfectly fine.

> Comments in Tcl work perfectly fine

I agree with what you're saying, but allow me to go meta.

Comments in Tcl are a bit strange. This works:

  # say howdy
  puts hello

But this does not:

  puts hello   # say howdy

For the later, you have to insert a semicolon:

  puts hello;  # say howdy

I get bitten by this time and time again.

Yeah I got bit by that too, once. But it makes sense. If everything is a command, then "#" must also be a command. Essentially like a NOOP.

Once I visualized it that way, I never got tripped up by that again.

And regardless of this fact, comments exist. They may not be as convenient as using // or // in C its friends, but they exist. To say that tcl does not allow one to know what a complicated line of code means is absurd.

> If everything is a command, then "#" must also be a command

In my mind, that's an argument that at least something shouldn't be a command. But perhaps it simplifies implementation enough to be worth it.

> To say that tcl does not allow one to know what a complicated line of code means is absurd.

Fully agree with you there. And for what it's worth, in the Tcl I encounter, there's rarely a line of code that complicated. Perl is often accused of being a write-only language. I think Tcl, if anything, is the opposite. It always seems readable.

A really nice article to get one started! thx

Another topic I would like to see introduced in a similarly friendly way is event loops. At my work I am dealing with some Tcl/Tk code that uses event loops and it's a bit of mental gymnastics to get my head around what gets executed when.

Although antirez made the sane choice of embedding Lua in Redis.

upvar always eluded me. TCL got heavy use at AOL in aolserver, a project that just switched from CVS to git http://www.aolserver.com/

Reply on news.ycombinator.com