Data Definition and Code Generation in Tcl (2014) [pdf] | Hacker News Reader

Data Definition and Code Generation in Tcl (2014) [pdf]

trs.jpl.nasa.gov

47 pointsgglitch22 comments

TCL is an often overlooked language. It has a lot of weirdness, but that's a symptom of it's age I think.

I really like it as a language for interactive command-shell type use cases, as opposed to languages like Lua. It feels much closer to a shell than a programming language. If you want some kind of interactive command shell, I would very much vouch for using TCL as opposed to writing your own from scratch. There is also a lightweight/embedded variation, Jim TCL[0] if that's more your bag.

It is an interesting choice for writing a parser in, but it makes a certain amount of sense. It's portable, battle-tested, and has surprisingly robust string processing facilities. I have also found it useful as a data format, but rather than writing a parser in TCL, I tend to prefer to define keywords to turn TCL into a DSL. Here is a trivial example I made up for this post: [1].

If you want to see some example code for integrating with C/Python, I have some here[2]. I had planned to give a talk on the subject but it fell through.

0 - http://jim.tcl.tk/index.html/doc/www/www/index.html

1 - https://rextester.com/FME19315

2 - https://github.com/charlesdaniels/teaching-learning/tree/mas...

I would say that most of the weirdness of Tcl is not caused by age, but because at its core it is nothing else than shell without all the tradional unix shell syntactic cruft. In the end Tcl is nothing else than shell with exactly two quoting mechanisms ("" and {}) and exactly two expansions ($variable and []), everything else is built as commands on top of this simple language.

I recently discovered that rewriting shell scripts into TCL works really well -- going from Bash to Tcl is a smoother transition than Bash to Python. The "exec" command even has support for redirections and pipes.

Reading this really reminded me of some of the fascinating/crazy things Tcl is capable of, all based on a simple programming model focused on strings. Like, want to pass a reference to a variable? Simply pass the name of the variable to the function and then do

    proc f {varname} {
        upvar 1 $varname var
        set var ...somevalue...
    }

The upvar “links” the local variable var to the variable named $varname in the scope of the calling function. Or maybe you are in dire need of a control construct that loops thrice? No problem:

    proc do_thrice {code} {
        foreach _ {1 2 3} {
            uplevel 1 $code
        }
    }

Okay, this does not handle break, continue or error propagation (you can do that also, but it would be a bit longer). You can use this like so:

    set text Go!
    do_thrice {puts $text}

The variable text is not visible inside of do_thrice; but the uplevel executes the code in the scope of its caller, so it can read and write to the local variables just like an ordinary loop.

Of course, in ordinary code you should mostly avoid this kind of stuff. Though apparently uplevel is used by NASA, of all people, so I guess it is fine!

More fun stuff: You use the built-in proc to define a new procedure, but you can replace it by your own, e.g. to insert profiling code at beginning and end. (Hint: If you try this, use uplevel when calling the built-in proc, to avoid namespace problems.) You can parse and rewrite code at runtime (of course) which I once used to insert “breakpoints” at runtime for my program to debug itself.

TCL is one of the few languages allowing you to dabble in to and one of the few that caught hook with me. It's friendliness then leads you down a path of voodoo and creativity.

  proc displayHackerNews {} {
  set hn "Hacker News" ;#Fill the variable (hn) with the string "Hacker News"
  puts $hn ;#output the variable ($hn) to terminal
  } ;#end procedure

  displayHackerNews ;#execute the procedure
   displays: Hacker News

If statements are easy

  set hn ""
  if { $hn eq "Hacker News" } { puts $hn } else { 
     set hn "Not Hacker News" 
     puts $hn 
  } ;#end if

Infinite Loops are fun

  set hn "Hacker News" ;#Set a variable
  while { $hn eq "Hacker News" } {
    puts "$hn" ;# print "Hacker News" to terminal in an infinite loop
  } ;#end loop

Multi-threading is a breeze.

  package require Thread
  set MyThreadID [thread::create { 
  puts "Hello from my new thread"
   thread::wait
  } ;#end code-to-run
   ] ;#end thread

And finally, a if-switch-threaded-loop

  set hn "Hacker News"
  if { $hn eq "Hacker News" } {set switch "hn" } else { set switch "" }

  switch $switch {
  hn { 
  set MyThreadID [thread::create { 
  proc displayHackerNews {} {set hn "Hacker News" ; puts $hn} ;#end procedure

  #set aCounter to zero, if aCounter is < 7 execute displayHackerNews procedure
  for {set aCounter 0} {$aCounter < 7} {incr aCounter} { displayHackerNews } 
   thread::wait
  } ;#end threaded-code
  ] ;#end thread
  } ;#end switch-case

  default {puts "It's still not hacker news"}
  } ;#end switch

To really show how much "everything is a string" in Tcl, I love the "sum a list" one liner:

    set a [list 1 2 3 4]
    set sumValue [eval [join $a +]]

[join $a +] produces a string "1+2+3+4" from the list {1 2 3 4} which you then evaluate to find the sum. Note that you do not need quotes around the +, tcl by default treats everything as a string.

Another example:

    set b a         #b now contains the string "a"
    set a 5         #a now contains the string "5"
    set var5 Steve  #var5 containst the string "Steve"

    puts $b      #write the value of b to the console: "a"
    puts $$b     #writes "5" to the console
    puts $var$$b #writes "Steve" to the console

Some of these things do not quite work this way, but they demonstrate some interesting peculiarities! The eval should be expr, as you are trying to evaluate a mathematical expression, not a command. Similar to the common footgun:

    set a 1
    puts [expr $a + $a] # Wrong!
    puts [expr {$a + $a}] # Correct

Also $$b expands to $a, not 5, as dollar expansion stops at the next non-alphanumeric-or-underscore character. This would usually be written as [set $b] (equivalent to [set [set b]] ), utilising that the set command outputs the value of the variable with that name. Similarly, [set var[set $b]] for the last line. Also, # is only a special character at the beginning of a statement, so comments have to follow a newline or semicolon.

In the “everything is a string” department, my favourite are block comments:

    if 0 {
        This is a comment.
    }

You can write any kind of comment in there, even special characters like $undefined or [undefined] which would cause errors in normal code. Only wrongly matched {} would be a problem.

mea culpa on the eval/expr. I did the comments inline for brevity.

I am confused about the $$ examples though, since swear I have worked on production code that literally behaves like that. Maybe they had some extra non-alphanumeric bits. This behaves the way I gave originally

    set b a
    set (a) 5

    puts $($b)

Edit: I found the production code. There was both

    puts [expr 3 - $$b]

and

    puts [subst $$b]

and the Steve example was actually two steps:

    set nameVariable [subst var$$b]
    puts  [subst $$nameVariable]

Yeah, that is another dark corner. I was not quite precise (I forgot this behaviour existed): $ expansion does allow for parentheses as well, so that arrays work. Meaning $($b) is the same as [set ($b)] which is element $b of the array in variable {} (empty string; that is the name of the variable). The docs [1] say:

> $name(index)

> Name gives the name of an array variable and index gives the name of an element within that array. Name must contain only letters, digits, underscores, and namespace separators, and may be an empty string. [...] Command substitutions, variable substitutions, and backslash substitutions are performed on the characters of index.

So that means:

    set b a
    set (a) a  ;# this sets element a of the variable {} to a
    puts $($b) ;# prints that element
    info vars  ;# all variables, note that {} is in there
    puts $($($($b))) ;# Of course this works...

[1] http://www.tcl.tk/man/tcl8.6/TclCmd/Tcl.htm#M14

Yes, and this everything goes makes it difficult to spot bugs. Perl byte-code compiled a program at start and was able to find a number of bugs (if the warnings option was enabled). If e.g. a variable is misspelled in TCL, you'll find out exactly when that particular line of code is reached. If that happens to be in the error path ...

I have it on good authority that this actually dates back to 2003! See the schedule of Tcl'03 [0], or the index page for this document at JPL's Technical Report Server [1]

[0] https://www.tcl.tk/community/tcl2003/schedule.html [1] https://trs.jpl.nasa.gov/handle/2014/7660

Interesting. I had no idea NASA used Tcl.

I've had people tell me in the past that TCL is used in all kinds of inconspicuous areas. This is the second time I've seen real world usage of it, the first being FPGA toolchains.

Two reasons: it was targeted as an embeddable application extension language (kinda like Python); and, thanks to Tk, it was a relatively easy way to make passable GUI interfaces on Unixes.

(There was also a very brief attempt early in the Web to use Tcl for Web development, as an arguably superior alternative to then-popular things like Perl CGI, but that didn't take over.)

What people should've been using for embeddable extension languages is a Scheme implementation, which, besides other merits, wasn't crippled by an evaluation model based on string expansion. I was in charge of one such extension language for a system that cost 5 figures a seat, and Tcl's string evaluation wasn't that up to representing objects and graph references well. (At the time, extension languages of major applications were often some simple/custom Lisp.)

Tcl was pretty neat, in how much more clever it was about some kinds of string evaluation than Unix shells, but a Scheme would've still been a better choice.

TCL is fat,slow,and lacks staying power. — (SIOD author) George Carrette, 1994

At least we could direct people who wouldn't see the light towards Python at that stage.

Another example is F5 load balancers. It's their embedded scripting for health checks, content transformations, etc.

There's a list of other places it's used here: https://wiki.tcl-lang.org/page/Who+Uses+Tcl

So it's not only sweetwater molecular dynamics that uses TCL but CCP4 as well. Nice! (Coot uses Scheme, though.)

I worked for HCI, which was healthcare.com, back in the day. Our entire UI and data layer, at the time, was TCL/TK. Things changed, of course, but TCL did have a lot influence in the Health Care/HL7 world...which marches on.

Yes, I recall Paul Emsley getting more than that right.

At one time the area was continually shifting around TCL, Perl, Python, not really finishing anything off. That was mainly one person's influence for some reason.

Oh wow I completely forgot about Cisco's IOS.

I've heard it's used a lot in EE design software (PCB design, or FPGA as you mentioned).

My experience with Tcl is as part of Altair's suite of mechanical simulation (FEA, MBD, etc) software, which is quite popular in aerospace and automotive.

It has its warts (associative arrays) but it's still a neat little language. EIAS actually works pretty well as an abstraction tool. I'd take it over VB (eg, excel VBA) anyday as a macro/automation language.

Good post on tcl by antirez, creator of redis: http://antirez.com/articoli/tclmisunderstood.html

Yeah, I use it in chemical process simulation and I pretend arrays don't exist. Dictionaries for everything, unless my software version is old enough that the Tcl version doesn't support them.

Reply on news.ycombinator.com