Extending Tcl
wiki.tcl.tk
wiki.tcl.tk
(I) design for size, ease of extensibiliy and embedding, and (II) doing threads right.
(I) became a feature offered by many other languages, but I think Tcl was the one that popularized it. This crown belongs to Lua now.
(II) was not adopted as much as it should have been. I am looking at you Python. Interpreter as a library with interpreter state fully contained in an object, no globals or statics -- simplifies threads immensely. Run different interpreters in different threads, no need for global lock, no need for serialization-deserialization to talk to other interpreters in the same address space.
When that happened JO went to found Scriptics, but it never got the traction it should (not helped being branded as "parasites" by Stallman)
Tcl is more pleasant than Javascript, but still has some horrible parts like poor support for structs, nothing really equivalent to a pointer, clumsy escaping, and (at the time) no support for dynamically linked extensions.
What I missed the most when doing Tcl was support for first class functions / closures.
JimTCL seems to adress many of the pain points, for example it adds closures garbage-collected OO. http://jim.tcl.tk/index.html/doc/www/www/index.html
That was Stallman's response to Sun bombastically pushing TCL as the official scripting language of the web, BEFORE Live Oak / Java was a widely known (or evangelized) thing.
At the point anybody started talking about a Java/TCL bridge, it was already all over for TCL becoming the "ubiquitous scripting language of the Internet".
Sun's unilateral anointment of TCL as the official Internet scripting language trigged RMS's "Why you should not use Tcl" message, which triggered the TCL War, which triggered Sun to switch to Java.
After the TCL war finally subsided, Sun quietly pushed TCL aside and loudly evangelize Java instead. The TCL community was quite flustered and disappointed after first winning the title "ubiquitous scripting language of the Internet" and then having the title yanked away and given to Java.
Any talk of bridges were just table scraps for TCL, the redheaded bastard stepchild sitting outside on the back porch in the rain, smoking a cigarette and commiserating with NeWS and Self.
Tom Lord's description of what happened is insightful and accurate:
https://web.archive.org/web/20110102015130/http://basiscraft...
>The Infamous Tcl War
>[...] Mr. Ousterhout had, a few years prior, developed Tcl while on the faculty of UC Berkeley - mainly, I think, to have a handy tool for other research and only secondarily as an experiment in language design. And he topped it off with Tk. Tcl/Tk took off in a huge way. It was easy to understand. The source code, written in Mr. Ousterhout's methodical and lucid style, was a joy to read. At the time, about the most convenient option for developing a GUI to run on a unix system was to write C code against the Motif toolkit - an ugly, expensive, and frequently disappointing process. With Tcl/Tk in hand, people started handing out new "mini-GUIs" for this and that, like candy. Tcl/Tk started to find application in some rather intense areas, like, for example, the "control station" software for some oil rigs. It was a smash hit.
>Meanwhile, I don't think I'm letting too many cats out of the bag here, the informal Silicon Valley social network of well placed hackers were quietly and unofficially circulating some very interesting confidential whitepapers from Sun Microsystems. One of their researchers, a fellow called Mr. Gosling, had dusted off a language he'd once led the design of called "Oak". Oak was originally intended for use in embedded systems. Its basic premise was that devices ought to be Turing complete and hackable, whenever possible. Oak's approach to statically verifiable byte-code comes from that origin. Mr. Gosling came out of Carnegie Mellon University and the attiude behind Oak was popular there. As one grad student had quipped a few years earlier: "If a light switch isn't Turing Complete I don't even want to touch it."
>In light of the rising star of web browsers, the folks at Sun conceived the notion of offering up a derivative of Oak to serve as the extension language for browsers. (It is probably worth mentioning here that Mr. Gosling was earlier well known for making one of the very first unix versions of Emacs.) Oak was re-named "Java" and the rest of its history is fairly well known.
>I've read, since then, that up to around that point Brendan Eich had been working on a Scheme-based extension language for Netscape Navigator. Such was the power of the hegemony of the high level folks at Sun that the word came down on Mr. Eich: "Get it done. And make it look like Java." Staying true to his sense of Self, he quickly knocked out the first implementation of Mocha, later renamed Javascript. This phenomenon of Sun's hegemony influencing other firms turns out to be a small pattern, as you'll see.
>Mr. Ousterhout was hired by Sun (later he would spin off a Tcl-centric start-up). The R&D; team there developed a vision:
>Java would be the heavy-lifting extension language for browsers. The earliest notions of the "browser as platform" and "browser as Microsoft-killer" date back to this time. Tcl, Sun announced, was to become the "ubiquitous scripting language of the Internet". Yes, they really pimped that vision for a while. And it was "the buzz" in the Valley. It was that pronouncement from the then-intimidating Sun that led to the Tcl wars.
>Mr. Eich, bless his soul, brute-forced passed them, abandoning Scheme and inventing Javascript. [...]
My heart stopped here for a while [0]. Are there any papers on Java as system language? I'd love to read more - as embedded developer, this really is out of my scope.
[0]: don't worry it does it 60 times per minute
https://en.wikipedia.org/wiki/Java_(programming_language)#Hi...
https://www.researchgate.net/publication/220938922_Writing_S...
They also created SunSPOT before RasperryPI, Arduino were even ideas, where beyond a basic layer written in native code, everything else was Java based.
Also Java is actually used as systems language in some scenarios, just not on your desktop.
When it is compiled AOT to be deployed on bare-metal scenarios such like the SDKs being sold by PTC and Aicas (just two examples, there are other companies)
https://www.ptc.com/en/products/developer-tools/perc
When it is used on Android Things to write user space drivers.
Like NeWS was the Network extensible Window System, so NeFS was the Network extensible File System, or NFS 3.0.
It was actually a great idea, just a wee bit before its time, and very poorly named and positioned!
For example: If you want to make a copy of a file on the server, you can send a PostScript program that runs in the kernel and copies the file locally on the server in the kernel with ZERO context switches, instead of sending it over the net to the client, then back from the client to the server. Even if you rsh'ed the user command "cp" on the server, it would still incur context switching, but if your copy loop was running in the kernel then it didn't need to switch in and out and in and out for every block it copied.
There are more examples of why it's a great idea in the paper.
This comparison of NeWS to AJAX also applies NeFS, which is like kernel NeWS with file operations instead of a graphics library -- it also saves you lots of user/kernel context switches even if you're not doing any networking:
https://en.wikipedia.org/wiki/NeWS
>NeWS was architecturally similar to what is now called AJAX, except that NeWS coherently:
>- used PostScript code instead of JavaScript for programming.
>- used PostScript graphics instead of DHTML and CSS for rendering.
>- used PostScript data instead of XML and JSON for data representation.
It didn't go over very well because the unenlightened philistines of the time couldn't get their head around an API to the file system that wasn't compatible with creat open close read write and ioctl.
http://donhopkins.com/home/nfs3_0.pdf
Network Extensible File System Protocol Specification
1.0 Introduction
The Network Extensible File System protocol (NeFS) provides transparent remote access to shared file systems over networks. The NeFS protocol is designed to be machine, operating system, network architecture, and transport protocol independent. This document is the draft specification for the protocol. It will remain in draft form during a period of public review. Italicized comments in the document are intended to present the rationale behind elements of the design and to raise questions where there are doubts. Comments and suggestions on this draft specification are most welcome.
[...]
Example: Copy a File
Make a copy of file (foo) called (bar). Both files exist in the same directory dfh. The request starts by looking up the filehandle for the file to be copied and creates a filehandle for the copy. The loop operator executes a procedure that copies the file using 1K reads and writes. It maintains a running count of the number of bytes yet to be copied.
% Copy a file
%
dfh (foo) lookup /foofh exch def % get filehandle for (foo)
dfh (bar) create /barfh exch def % create filehandle for (bar)
/bytes foofh getattr /fsize get def % get size of (foo) so we know how much to copy
/offset 0 def % initialize offset for (bar)
{
/data foofh offset 1024 read def % read up to 1K from (foo)
barfh offset data write % write up to 1K to (bar)
/bytes bytes 1024 sub def % decrement byte count by 1024
bytes 0 le { exit } if % if it’s < 0 then we’re done
/offset offset 1024 add def % increment offset by 1024
}
loop
barfh getattr 1 encodereply sendreply % return the attributes of the new file to client.http://donhopkins.com/home/archive/NeWS/Comp.protocols.nfs.N...
From: brent@terra.Sun.COM (Brent Callaghan)
Newsgroups: comp.protocols.nfs
Subject: NeFS Protocol Spec Available
Keywords: NFS version 3
Date: 13 Feb 90 01:57:31 GMT
NeFS is not NFS. It started out as an NFS protocol rev
(NFS version 3) but its protocol is a radical departure
from the NFS's remote procedure call model. In order
to avoid confusion with NFS we're calling it NeFS for
now - Network Extensible File System (the similarity of
the name to a window system protocol is entirely intentional).
A draft spec for this new protocol is available via anonymous
FTP from titan.rice.edu (128.42.1.30) as
/public/nefs.doc.ps
The file is ~250K of PostScript. It prints 52 pages.
The spec is not available in any other format.
Comments, suggestions, flames etc are welcomed. Direct them
to "nfs3@sun.COM".
Thanks
Made in New Zealand --> Brent Callaghan @ Sun Microsystems
uucp: sun!bcallaghan
phone: (415) 336 1051
From: mo@messy.bellcore.com (Michael O'Dell)
Newsgroups: comp.protocols.nfs
Subject: RE: NeFS protocol
Date: 23 May 90 11:57:34 GMT
Organization: Center for Chaotic Repeatabilty
Dave Clark, Internet Architect, just gave a talk here at Bellcore and
he was lobbying for exactly the kind of approach being proposed by the
new NeFS protocol - don't send packets, send programs!
I do recommend you read the NeFS document and think about it hard
before you jump to any conclusions. The proposed model is novel, and
goes a long way toward moving us to "action at a distance" instead of
"remoting local operations." This turns out to be vitally important if
networks are to scale with latency, since, as Dave Clark so eloquently
showed, for large networks (ie, country-sized), as bandwidth goes to
infinity latency goes to 30milliseconds, and you can't do anything
about that, because you can't change the speed of light. So, to do
very well in the face of that, you must avoid round-trip delays like
the plague. Sending a program to the server can save many round-trips
for many operations.
Anyway, it is interesting reading. Particularly in light of Dave
Clark's talk.
-Mikehttp://www4.cs.fau.de/Projects/JX/
https://www.cs.cornell.edu/Info/People/chichao/sip99.pdf
https://www.azul.com/products/vega-processor/
Although I don't keep them bookmarked, I looked into my collection of real-time or embedded GC's to find these that you might feel inspired by or see as potentially useful. :)
https://www.diva-portal.org/smash/get/diva2:20899/FULLTEXT01...
http://www.vmars.tuwien.ac.at/documents/extern/3017/diss_puf...
https://researcher.watson.ibm.com/researcher/files/us-bacon/...
The primary issue is that Tcl was a language designed on the wrong side of the memory cost divide. Many of the languages designed with "memory and CPU are expensive" got swept aside because newer languages that didn't make that assumption had a lot more features.
What is sad is that Tcl didn't STAY SMALL. In an era of 32K ROM/4K RAM microcontrollers, we don't really have a well-featured scripting language that works down there.
It also didn't help that Tcl went into a period of instability right as the Dot Com Boom/Crash went full tilt whereas Python, for example, was stable from 1997-2000 (A LOT of people standardized on 1.5 for a loong time)
And, as smart as he is, John Ousterhout was one of the big coding problems and getting him out of the codebase helped Tcl dramatically.
What about Forth? http://www.forth.org/
I had really high hopes for Lua. But 1-based indexing when you're interfacing to C is just not flying. Apparently I'm not alone as Lua comes in at #9 on the "Dreaded" languages list:
https://insights.stackoverflow.com/survey/2017#technology-mo...
That does not stop a lot of people. Not that it matters, because Lua will not run in 4KiB RAM.
Threaded code is a compiler/interpreter implementation technique. That makes as little sense as saying you won't use C, because you are "trying to solve a problem, not play intellectual games for how to create static single assignment code."
Instead of being an armchair programmer, you should actually try to learn those programming languages.
Dormancy and stagnation can have many causes: author hiatus, the code clearing some threshold feature-complete, obsolescence by competing tech that solves similar problems. And it seems Tcl has managed to stay relevant through a combination of usefulness (like its standard lib, or Tk) sufficient paradigm difference to tech that came later, in some cases future tech building on top of it instead of replacing it wholesale, willingness to fix common gripes in the core (numbers, unicode, binary), and a developer-focused (instead of consumer-focused) approach to packaging. It's probably helped by being a captive scripting language on some embedded platforms too.
Languages that became big and popular in later years faced demands around the distribution of third-party packages. Perl became an early pioneer with a community repository, but later languages had to cope with the expectation of using a language-specific package manager that would download from the official repository, and automagically make packages available globally. Tcl encouraged user contributions in a less formal way [1] -- develop programs (code that uses builtin commands) or extensions (code that defines new commands) for your own use, and make them available if you so wish. This minor difference in attitude creates an ecosystem that feels different from a language package repo, where thousands of packages of varying quality sit, the long tail of them effectively unvetted.
[1] https://www.tcl.tk/community/tcl2017/assets/talk93/Paper.htm...
aside: Everything tcl just worked for you on macOS? Mac having 8.4 rather than 8.6 drove me nuts.
I helped work on the 2nd edition of this book, which I think is still fairly relevant: https://amzn.to/2IB9OyR
It is probably one of the best languages to extend. Sadly it is also one of the slowest.
Early Tcl was slow - but it was being compared to pure C. Recent Tcl is as snappy as any HLL.
In terms of pure runtime performance (CPU efficiency I guess), do you know of any reasonable benchmarks comparing TCL to say, Python and Ruby? I wonder if they might all be in the same ballpark?
It was a great experience and we had quite a few in-house tooling that I only saw years later in Rails.
Those issues lead us to become early .NET adopters as Microsoft was searching partners for getting their products ready to ship when .NET 1.0 would be announced to the world.
Since then I only use scripting languages for shell scripting.
Hm.
In fact, the .NET re-rewrite was quite successful and many of the people involved on that team went on to create a very successful product based on the field lessons we had with the Tcl version.
And in the end we gave up for .NET.
More often then not, the problems I was involved with resolving on emergency escalation (line down) calls was on the SAP side of the world and never in the integration code in TCL and C.
The biggest drawback with the TCL implementation from our application vendor was that they did not implement multi threading for TCL and instead implemented a node.js like event loop. As SAP calls were locking, this did cause locks for all other TCL calls.
Who’s event-loop is like who’s? ;)