How I Recovered Millions of Transactions Using Tcl (2016) | Hacker News Reader

How I Recovered Millions of Transactions Using Tcl (2016)

activestate.com

69 pointsgglitch28 comments

Not TCL, but my Perl war story was I worked for a printer company. They had somehow gotten someone to record ink cartridge information into external data files. INI files to be precise, one INI per record.

They were literally spending 9 weeks opening about 15,000 INI files and pasting them into an Excel spreadsheet manually.

I was rather silly and made the claim that Perl was designed for this sort of thing, they challenged me and I felt my honour was at stake, so I whipped up a program to do it for them one night and brought it in the next day. I ran it on the INI files, and in about 20 seconds it had put all the data into a CSV file.

They were astounded that I had taken 9 weeks of tedious, error prone data entry and processed it in less than a minute. I just figured it was an interesting thing to do, and told them to keep it quiet as this was really outside my area, but they made me employee of the quarter anyway.

how much did you charge for it?

Nothing.

This is the curse of Windows - non-technical users assume that everything you can do with a computer must be done via the UI.

This is perhaps not surprising - in the name of "ease of use" Windows works very hard to shoehorn you down the point and click route.

Visual is more natural, simple and rich way of cognition for human brain than symbolic expressions, there's nothing wrong with it. What's somewhat pity is the fact all the attempts to create any 'visual programming' tools failed miserably. Although we could easily imagine a tool that allows to point a field in any formatted text like INI file, and point the column in Excel where the values should be inserted, and automatically build the regular expression to parse the bunch of such files - that's not visual programming in any way of course, but if such a tool could be extensible enough, and extensible by the same visual way, I don't know but maybe some significant part of users should be interested enough.

TCL is pretty awesome. I hadn’t used it since the late 90s, but in a recent project I ended up integrating it into a Go project we’re building, and it was great. Extraordinarily easy to do, and very powerful.

I really need to write a blog entry about it. It’s now a valuable tool in my toolbox.

Tcl is bananas. On the one hand, its approach to structuring data (cram everything into strings, including procedures!) is... idiosyncratic and not what I'm used to. On the other, it seems to work hand in glove with the "Unix Way" (for whatever that's worth), and I've never seen a faster way to get a GUI up and running than Tcl/Tk. Not Visual Basic. Not even Smalltalk.

"not what I'm used to" != "bananas"

But it is bananas.

https://vanderburg.org/old_pages/Tcl/war/0018.html

Tcl was my first language, and I enjoy it very much.

But I hope I will never have to write anything serious in it. And I write Perl for a living.

It's a bit idiosyncratic for sure. For one I always thought why the hell would you design a language and not use "=" as an assignment operator. On the other hand it's not a good reason to reject a programming language when it has a lot of other good features.

> why the hell would you design a language and not use "=" as an assignment operator

That never bothered me.

One thing that bothered me was how hard it is to write Tcl code without errors.

After the introduction of the special syntax {∗} in Tcl 8.5, correct list concatenation went from:

  set concatenation [eval [linsert $list1 0 list] [lrange $list2 0 end]]

to:

  set concatenation [list {*}$list1 {*}$list2]

Mind you, Tcl 8.5 is quite late in the history of the language, and the special syntax is still not super obvious.

This example was taken from:

https://stackoverflow.com/questions/5124185/what-does-do-in-...

Another thing that bothered me was the use of upvar/uplevel as a substitute for pass-by-reference: While upvar/uplevel is incredibly powerful (allowing you to evaluate code in any parent scope), this opens up for macros. The upside is super powerful macros. The downside is readers of your code will not know what's going on unless you stick to a set of conventions. I've experienced the same problem in syntactically saturated dialects of Lisp. Compare this to Scheme's 'hygienic macros': https://en.wikipedia.org/wiki/Hygienic_macro -- in Tcl, you can't look at an arbitrary piece of Tcl code and know anything about what's going to happen.

> why the hell would you design a language and not use "=" as an assignment operator

That's one of my favorite things about Tcl. It just makes so much sense. Testing equality is analogous to testing inequality. It's a complex and context-dependent operation. Binding a value to a symbol is a completely different type of operation, and there's no reason for it to look similar. Keeping arithmetic in its own little space makes everything else so much simpler.

Many of the most interesting new languages I've seen which don't come from the C/Unix school use something different for assignment, e.g., Factor (Forth), Clojure (Lisp), etc.

Nor is it just the new kids - ":=" for assignment goes back to Pascal and Smalltalk (IIRC Smalltalk used a funky left arrow, like R but a single char, but also allowed := because who has a funky left arrow on their keyboard?)

Like Lisp, one of the oldest languages ever?

> On the one hand, its approach to structuring data (cram everything into strings, including procedures!)

Tcl has not actually "crammed everything into strings" since circa. 1997 with the release of Tcl 8.0.

Now, you can obtain a serialized string for most every data structure if you want one, and for quick serializing to a file or debugging this is useful. But the language has not operated solely with strings for everything for a very long time.

Rebol would like a word with you :)

http://www.re-bol.com/rebol.html

Makes Tk seem painful to me. There is a new "Red" language in the works to supercede Rebol and be a true full-stack language with an actual native GUI for each OS instead of the single Rebol GUI system which looks the same on every system.

Python is a great way to run Tcl ;-) Seriously though, Python's Tcl bindings are done at the C level and provide great interoperability.

At a previous job I utilized these low level bindings to create higher level bindings. Used reflection of both Python and Tcl to expose themselves to each other. Was able to seamlessly use Tcl objects from Python and vice versa.

While browsing a Ruby codebase’s config.rb recently, I saw several proc commands followed by bracketed subcommands and wondered if it was for some reason Tcl embedded into a Ruby script. Could that be an example of what you’re describing here? What’s the use case for that?

> While browsing a Ruby codebase’s config.rb recently, I saw several proc commands followed by bracketed subcommands and wondered if it was for some reason Tcl embedded into a Ruby script

The proc method in Ruby takes a Ruby block, which is delineated either by brackets or do...end. it has nothing to do with embedding Tcl.

The article says "Tcl’s innate asynchronicity". Does anybody who knows Tcl care to tell more about this?

I wouldn't call Tcl "innately async" like, for example, Erlang.

Tcl is a command-based, imperative language. Tcl has threads.

They work mostly like you'd expect: shared variables, mutexes, condition variables.

Threads can send script code to another thread using thread::send -async and thread::broadcast:

https://www.tcl.tk/man/tcl/ThreadCmd/thread.htm#M17

Tcl does let you write DSL-like libraries, and because you can easily hotswap code (being homoiconic like Lisp), it also works well for distributing your execution environment across several network nodes.

But the language itself isn't "innately async". The thread model isn't based on message passing, CSP, Pi calculus, or any other strong idea. I always thought of it as "string-focused Lisp".

Tcl has built-in event loop functionality. You can put watches on file channels, network sockets or synthetic scripted channels, start the event loop, and have a completely event-driven program.

Writing a simple web server, for example, is trivial.

Tk accesses the C-based event API to turn things like button clicks into events. Python (and Ruby?) ships with Tcl in order to make use of Tk.

There's a thread extension that makes it easy to spawn separate threads which can inject events into the event loop, making hybrid event/thread programming simple.

This is right. It's "innately async" in the same way as Node: there is a built in event loop, and just about anything that could block (file I/O, GUI input, etc.) can be accessed through async functions instead.

One nice thing about Tcl vs. Node: Tcl doesn't crash if there's an error in one of the asynchronous call stacks. The Tcl interpreter catches the error and passes it to a (customizable) background error processor, and keeps going.

That's right. My first reaction after learning Node was that its Tcl/TK done poorly but with a faster runtime. Tcl/Tk had first class event loops from the beginning (of time), later it acquired coroutines as well.

The best part for me, however, was that in Tcl you can have multiple interpreters in the same process, each in its own native thread if you want. No GIL, no need to pickle/unpickle to talk the other interpreter. Had Python adopted this model, using multiple cores would have been so much nicer.

That said, the mantle has passed on to Lua, but I still harbor a soft corner for Tcl.

How different are tcl and node from select, poll, and epoll?

I know tcl and node are runtimes - but what do you get extra in their event loops compared to standard event loops that Linux offers?

It is difficult to compare languages with operating systems on which those languages can be executed/evaluated.

On systems that implement them, Tcl can use select, epoll, kqueue for its event notification implementation, as well as pthread_cond_broadcast for threads (if built with thread support).

take a look at:

generic/tclNotify.c - https://core.tcl.tk/tcl/artifact/8804164706c8f2cb

unix/tclEpollNotfy.c - https://core.tcl.tk/tcl/artifact/892b68fa5a7c4c87

With select/poll, you have to write your own dispatcher, which means you have to care about details like whether you need round-robin to prevent early things in your fdset stopping later things being processed, etc. Also, many of the things you'd like to wait on aren't file descriptors, so you have to do extra work to make them fit. My bugbear for this is pthread condition variables, but it's also true of most GUI toolkits, which expect you to use some variation of while(Toolkit_GetEvent(&event)). You can work around these, of course, it's just mildly annoying extra work.

Reply on news.ycombinator.com