Picol: A Tcl interpreter in 500 lines of code | Hacker News Reader

Picol: A Tcl interpreter in 500 lines of code

github.com

131 pointstosh72 comments

Quite a few years ago, I was looking for a way to program microcontrollers via an onboard interpreted language, and happened on the picol code. I still have my version somewhere. It was delightfully straightforward and easy for an amateur to modify. So I first got it to compile on a Teensy, then added a REPL, floating point, and a few functions to do things like flip the GPIO bits and read the ADC.

Today of course there are things like CircuitPython, and I enjoy using it too, but it's such a beast that it's really impenetrable for someone at my skill level if I want to add low level functions.

Thanks for your comment! Maybe it's time for picol2.c, same amount of lines of code, but floats and [expr]. I'm going to use a few hours today to get there. Such small changes would make it more practical for real use cases where one wants to retain understandability.

Released.

Sadly it seems the blog post that was released in the past is no longer available, but the wayback machine has a copy

https://web.archive.org/web/20220303135439/https://oldblog.a...

That provides background about the constraints/limitations in this code.

Restored. Recently I migrated all my stuff into a new server and missed to migrate the /pages of the old blog. Thanks.

Classic antirez. Does more in 500 LOC than most frameworks in 50k.

Yes. His work is really inspiring, eg. the text editor less than 1000 lines, tiny language in less than 1000 lines etc. When I read this on HN, I converted his text editor to my hobby programming language [1] (330 lines), and then wrote a chess engine with terminal UI [2] (400 lines), Tetris clone [3] in 140 lines. I also have a QR code generator and parser [4] (700 lines, still in Java only), a PDF generator (200 lines), and now also a tiny programming language [5].

[1] https://github.com/thomasmueller/bau-lang/blob/main/src/test... [2] https://github.com/thomasmueller/bau-lang/blob/main/src/test... [3] https://github.com/thomasmueller/bau-lang/blob/main/src/test... [4] https://github.com/thomasmueller/bau-lang/blob/main/src/test... [5] https://thomasmueller.github.io/bau-lang/at.html

best thing about TCL is easy syntax and that everything is a string :) Unique and simple and easy language with very slow changes.

Something like Python in good old days of 2.x before young internet javasceipt devs started pouring A LOT of new features to the language (feature creep).

Nowadays Python is so complex and flooded with ex C, C++, Java, JavaScript, Haskell programmers adding so many features, so fast that it's impossible to follow and understand them :(

Languages should not evolve on that rate. No time to master it :(

/rant

> best thing about TCL is easy syntax and that everything is a string :)

What? That's the worst thing about TCL.

I love it. Living proof that languages doesn't need forced types.

Data is Data. It's kinda object programming as visioned in the 70".

So easy and trivial language.

> Living proof that languages doesn't need forced types.

The opposite surely? TCL is practically dead, and only lingers on in the EDA industry (which has zero taste). Virtually every successful language today has at least a few different basic types for numbers, arrays, strings, maps and so on.

Tcl has these types as well. They are special strings.

Number: string of digits. List: strings separated by space. Dictionary: key value pairs in a list.

No they aren't. Everything is a string. The runtime does some optimisation where it actually does use different types under the hood but they don't change the semantics.

If you try to pass a list into a function expecting a string it will just treat the list as a string. No type error or anything, not even at runtime.

CMake is the same. Terrible design. Real types are useful.

TCL's typicing discipline comes from shell languages and awk, which are decidedly not dead. The overwhelming majority of programmers have less than no taste, so language usage rates in general don't really mean much.

Well, if you use flightaware to track flights, you're using tcl. The vast majority of their codebase is in tcl.

If you place a sports bet online, odds (!) are that it will run through tcl in its business logic. I may even have written some of it.

It's only the interface that is of a string. An array is a list of strings with space between, but only for the programmer. The runtime is free to use an actual array.

I'm aware. It's awful because of the implications for the programmer, not the runtime.

It depends on the use case. For instance you open a socket, write there any value you have without serialization, read it in the other side, data transfer done.

The convenience of not having to marshal data over a network is certainly a use case. But I'll admit that two of the worst programs I ever saw were written in Perl and TCL. Somehow just a big jumble of inscrutable regexes.

When "everything is a string" then you have no choice but to literally treat everything as a string. Painful. Very cool project though.

That sounds more like someone who over-uses regexes - the developer is the problem, not the language

TCL has a lot of options for manipulating strings - take a look at just the built in string command for a start: https://www.tcl-lang.org/man/tcl8.4/TclCmd/string.htm

I have seen terrible code in Python, C, and more.

The objections to tcl I see most often seem to reflect an outdated or incomplete understanding of the language and its capabilities.

As with many interpreted languages, maybe it's just too easy to iterate interactively with code until it "just works", rather than taking the time to design the code before writing it, leading to perception that it is a "write-only" language.

However, despite its reputation, even Perl can be written in a way that is human-readable. I agree with you that it is more a reflection of the programmer or the work environment than of the language itself.

> When "everything is a string" then you have no choice but to literally treat everything as a string.

As someone who has been developing tcl almost daily for more than 30 years, both object oriented and imperative code, I have not found it necessary to think this way.

Can you explain what leads you to this conclusion?

That's certainly a take...

I agree, so much for the benevolent dictator idea.

There's certainly something to be said for stable, uncomplicated and minimalist tooling that wont evolve out from under you and leave you with something that won't just-work five years from now.

I guess that's why Tcl is so popular in the EDA arena. I can stick some custom JTAG tooling in a Cyclone II design and talk to it by Tcl-scripting the 15-year-old software - and be confident that the same code (both in the FPGA and on the host computer) would work with the latest software and a current device.

Having said that, Tcl's not entirely free of compatibility and fragmentation frustrations: I sometimes wish that OpenOCD used full-fat Tcl rather than JimTcl, just so that I could make use of Tk. Being able to plot a histogram of data collected from the FPGA or make clickable buttons to trigger events is very useful.

Tcl has stopped being everything a string with the release of Tcl 8.0 and bytecode engine.

> In earlier versions of Tcl, strings were used as a universal representation; in Tcl 8.0 strings are replaced with Tcl_Obj structures ("objects") that can hold both a string value and an internal form such as a binary integer or compiled bytecodes.

http://www.ira.inaf.it/Computing/manuals/tcl/man-8.0/Changes...

I remember this quite well, because as part of the core team tasked with writing native C extensions, the migration to Tcl 8 had quite an impact on our code.

I learned Python with version 1.6, and have a few O'Reilley books proving the point the language wasn't really that simple, those that never bothered reading the reference manuals end-to-end though it was.

There have always been attempts at caching non-string values in Tcl for performance gains, but the semantics are the same. The value’s types are whatever the operator requires and an error when not compatible. If, internally, $x is an integer, and you pass it to a string function, well, $x gets temporarily turned into a string. Dynamic with “weak” types.

Which is exactly the quoted text.

No. The quoted text says nothing about the value semantics, only the representation…

Well, if you followed the link under the quoted text, you would get the next sentence,

> The new objects make it possible to store information in efficient internal forms and avoid the constant translations to and from strings that occurred with the old interpreter.

Huh? I know how it works. My point is different than the representation.

The worst thing about tcl is that it exists... and it's still the language of choice for EDA tools.

I'd be curious to know what your experiences with Tcl have been, to better understand why you feel this way.

It's a combination of cumbersome syntax and the vendor's poor API design and execution environment.

For a syntax example, compare python to Tcl. In python, there's a compact array lookup operator, and dictionaries allow string keys:

  mydata["channel"][2]

Tcl, however, requires a function call to access an array index, resulting in nested function calls with magic numbers for the simplest expressions:

  lindex [lindex $mydata 35] 2  # Tcl

Then there isn't any static type checking so you have to run code to see if it would work; the shape of a return may surprise you. Connecting to hardware may be your only opportunity to exercise certain code paths.

So for any non trivial task, Tcl becomes effectively write-only.

Some tools allow you to export Tcl scripts that automate tasks like project creation to help you avoid tracking intermediate objects in source control, but because the Tcl cwd path can change throughout a script, they export every source path as a hard coded absolute path, including whatever developer's home directory it might be residing in. This is not suitable for source control (because it would preclude building on any other machine) so then you have to post process these scripts before tracking them. But since it's Tcl there aren't good scriptable tools for programmatic Tcl AST manipulation, so you have to modify these files with some sort of a state machine, looking for various sentinels to trigger the various string replace behaviors you need. It's not all Tcl's fault, but if a vendor is only using Tcl for automation and you need automation, you have a battle ahead of you.

Outside the language, you'll often find that your runtime tool locks the UI thread to run your scripts, so long running operations are impossible to pause or cancel. And I've never seen a vendor expose adequate objects to allow full control, so often, the critical actions you hoped to automate are only possible through the GUI.

In the 90s, an integrated Tcl interpreter was a good sign that your vendor cared about developers. Today it's more a sign of neglect.

> Tcl, however, requires a function call to access an array index, resulting in nested function calls with magic numbers for the simplest expressions:

I do understand how that one can be frustrating to a newbie, but I can't fault that implementation choice when thinking about Tcl as a "Lisp but it's strings".

> Then there isn't any static type checking so you have to run code to see if it would work; the shape of a return may surprise you. Connecting to hardware may be your only opportunity to exercise certain code paths.

This, though, yeah. I feel this in my whole body. I haven't checked out Tcl 9.0 to see how it's changed, but Tcl 8.6's abbreviated compilation phase definitely doesn't leave any room for type much type inference ahead-of-time.

> sentinels

Tangential, but man I wish Tcl had `gensym` out of the box. That alone was enough to start my own programming language over a decade ago.

Everything else, I largely agree with. Short of a hard fork or the Tcl team splitting their time between maintaining for the sake of EDA vendors, and trying different approaches, I don't see much changing.

I really appreciate the time you took to share your experiences here.

You can just use

    lindex $mydata 35 2

for indexing nested lists, btw. Same for nested hash tables:

    dict get $mydata channel 2

Mixing the two does get ugly with nested functions, granted.

My experience with Tcl is 30 years of using EDA tools, all of them equipped with this language from hell. Luckily, I've been to restrict the scripting part to the minimum. I'm sure that if you know the language well, you can see the foundational model that's behind it, but for occasional users, there's nothing predictable about it.

I couldn't tell you exactly all the thing that are wrong with it. Let's just say that a language where this is ok:

puts "Hello"

But this is not:

puts "Hello" # A comment

shouldn't exist.

Similar amount of experience with tcl here (since dc_shell-t was released). I love working in tcl. It can be expressive, redable, sufficiently performant, and makes it easy and fast to develop and test new features and functions in even very complex EDA design flows with 100k lines of code.

I find it curious that so many of the criticisms of tcl are that it doesn't have some feature or behavior of some other language. That has never stopped me from accomplishing what is needed with clear, elegant code.

Rule #10 in the Dodekalogue [0] explains why your preferred comment style is not supported: a command is not expected immediately after the last argument of the previous command. Terminate the command per rule #1 and it is perfectly fine:

puts "Hello" ;# A comment

Isn't that the same number of characters one would type to start a javascript comment? //

Almost any programming language can be beautiful in its simplicity if you work with it on its own terms. tcl doesn't deserve the hate.

[0] Dodekalogue is the common name for the twelve rules that define the syntax and semantics of Tcl. https://wiki.tcl-lang.org/page/Dodekalogue

Would you prefer dc-shell scripting?

(I still remember the excitement of EDA tools _starting_ to support TCL and replacing the custom scripting engines they used to have)

I remember we had users at a chip foundry who were super deep into TCL. Is it still used anywhere else? Would you use it for a new project if you for instance already knew Python or Lua?

It's still the lingua-franca of ASIC/FPGA/Simulation, especially for scripting the tools.

I think it's slowly being replaced there by Python, but it's very slow.

> but it's very slow

The process of replacing Tcl, or Python itself?

Antirez (the author of this interpreter) uses it for the Redis test suite.

Personally, I know Lua and Python very well but I still used TCL a few years ago for something very specific: using ODBC on Windows. I gave more details on the Lua mailing list here: http://lua-users.org/lists/lua-l/2021-01/msg00067.html

That was interesting to read. So in your case it came down to ecosystem depth, especially on Windows - where there was a TCL solution to talk ODBC, but the equivalent Lua functionality had issues on Windows.

It is still used for operations procedures in, at least, European space industry. E.g., in mission control systems from Terma (CCS5 and TSC).

Tcl/Tk to this day is the best tool to make GUI frontends for CLI applications. Besides that, in the past I used it in production for intranet database entry applications. It has since been replaced by Flask. In my opinion the higher complexity of the Flask version is no way justified and we should have kept the much simpler ~2k LOC Tcl/Tk solution.

>Tcl/Tk to this day is the best tool to make GUI frontends for CLI applications.

Why? Please elaborate. I've heard others say this, but would like to know more.

>Besides that, in the past I used it in production for intranet database entry applications.

GUI apps?

> Why?

CLI applications typically read text from stdin and write text to stdout. The tcl model of "everything is a string" makes exactly the right abstraction to create GUI frontends for CLI applications rapidly and keep them simple at the same time.

thanks.

> Why? Please elaborate.

Tk's GUI object model is sitting at a reasonable maxima between trivial to make use of vs. triggering the events necessary to make the GUI active.

Small example. You want a button on your GUI that triggers a procedure called "process" when it is clicked, this is the code you need (if you are fine with the remaining defaults) (and assuming you used 'pack' as the geometry manager):

    button .process -text "Process Entries" -command process
    pack .process -side top

And a fully active GUI button will now appear at the bottom of your Tk window, with a label of "Process Entries" and when you click it, a Tcl procedure named "process" will be executed.

thanks.

In the 1990s, entire GUI applications would be written in Tcl/Tk -- not just the frontend -- the entire thing.

What is the best and easiest to install binary distribution?

I was looking into this for v9 and gave up on finding a binary which was:

- easily found

- agreed upon as the one to use

- available for and easy to install on all three platforms

Should I look again?

https://github.com/teclabat/tcltk-binaries

For jimtcl (the production-ready minimalistic tcl implementation which antires started (with full closures & gc'able lambdas) - probably an outgrowth of picol), you can get static "(some) batteries included" binaries at:

https://github.com/dbohdan/jimsh-static

For full tcl with the kitchen sink, bathtub & shower included, for android (hence the name) & regular Linux:

https://androwish.org/home/home

We've got Check Point Firewalls at work, and during updates I see lots of .tcl in the web UI.

I'm looking to use TCL for an embedded platform, as the scripting shell. Its simplicity and small size is key, as is its easy integration with C.

It is at the core of the SQLite test suite and also a first-class citizen in interfacing to it.

By the way, I'm guessing that Tcl was (is?) big in EDA because John Ousterhout also wrote the EDA tool Magic.

https://en.wikipedia.org/wiki/Magic_(software)

Tcl's first "release" was in 1988 and Magic was released in 1984, but it seems like John Ousterhout then wrote Tcl to have a common interface across tools [https://web.stanford.edu/~ouster/cgi-bin/tclHistory.php]. It still is a big thing in EDA, because (in theory) it lets a chip designer run their designs by tools from different vendors

I like tcl as an alternative to shell for any non-trivial scripts.

And its coroutines + built in event loop makes for really nice seamless async style programming.

i like calling a function f without arguments simply as

f # first word in TCL command line is a command

rather than

f ();

if there was also a function called g, i like to do things like

set a f

$a # calls f

so i can now

set a g

$a # calls g

cool i think.

The MacOS hex editor HexFiend uses Tcl for its template language.

> https://github.com/HexFiend/HexFiend/blob/master/templates/T...

I recall my time in satellite control systems had a lot of TCL code.

The previous company I was working at had quite a lot of TCL code for their back end logic. Betting sector, well known in US and Europe. They still actively hire people to maintain the codebase. Rock solid code, no surprises, was able to handle tens of thousands of concurrent bets.

JimTCL has more features and it's almost as small.

You probably know but others might not: they have the same author.

Yep, but Jim has been maintainer over and expanded.

Altough the SDL2 bindings are very alpha.

haha golden comment

https://github.com/msteveb/jimtcl/blob/master/README#L144

Nice idea.

picol is ideal for writing other language, for example esotheric language

Reply on news.ycombinator.com