Lisp, Smalltalk, and the power of symmetry (2014)
insearchofsecrets.com
insearchofsecrets.com
[1]: This book introduces the phrase structure in a shockingly LISP-like way. https://www.amazon.com/Grammar-Frank-Palmer/dp/B000S5VSAS
A downside I see with S-expressions – you often end up with something like (FOO BAR BAZ), where FOO identifies the type of thing, and BAR and BAZ are parameters/slots/etc – but you have to remember the (sometimes quite arbitrary) order that BAR and BAZ go in.
Some Lisp family languages (such as Common Lisp, Clojure, Scheme with SRFI 88/89) support keywords (aka named parameters), so you can do something like (FOO :bar BAR :baz BAZ), which is considered equivalent to (FOO :baz BAZ :bar BAR). However, this is somewhat of a late development in the Lisp tradition, the status of its adoption is mixed, and (arguably) it is moving away (even if only by a little bit) from pure S-expressions. When people say "S-expressions", they normally aren't thinking of CLOS.
I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. The downside of JSON objects, is their keys are strings, not symbols, and they don't have any kind of "class slot" (some people make up conventions here, for example a reserved property name such as "$class", but those conventions are not uniform or standardised). Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object.
(define xs `(foo (bar ,1) (baz ,0)))
(second (assoc 'bar (rest xs))) ;; 1
(define (prop xs k) (second (assoc k (rest xs))))
(prop xs 'bar) ;; 1One is that keywords are simpler syntax – `(foo :bar 1 :baz 0) is easier on the eyes than its association list equivalent.
Another is that their native implementation (linear search) can be a performance drain if you use them too heavily in an application (especially if you have too many keys). It is a data structure which optimises for insertion over retrieval, but most applications retrieve much more than they insert. The ability to "shadow" (have multiple bindings for the same key, with the first being used and the subsequent being ignored, but potentially becoming active again when the earlier binding is removed) is useful in some cases, but more often than not an unnecessary feature, more likely to induce bugs than be genuinely beneficial – "shadowing" by accident rather than intention.
A sufficiently smart runtime could optimise around some of these issues (this looks an association list, I'll use some optimised data structure to store it, but fall back on a more traditional cons-cell based storage if you try to do with it things the optimised data structure doesn't support.) But, having an actually separate data type enables equivalent performance optimisations, without anywhere near as much "smartness" in the runtime (which can be difficult to understand and maintain.) Plus, that kind of "smartness" can lead to difficult to understand performance regressions (why does adding this new code suddenly make completely unrelated parts of the application a lot slower?)
Property lists and keyword arguments aren't tied to CLOS, though of course CLOS uses them for the flexibla constructor arguments passed to make-instance.
A Lisp can use an alist to represent a set of key-value pairs, and is easy enough to shadow values there:
(defvar original '((a . 1) (b . 2)))
→ ((A . 1) (B . 2))
(assoc 'b foo)
→ (B . 2)
(defvar bar `((b . 3) ,@foo))
→ ((B . 3) (A . 1) (B . 2))
(assoc 'b bar)
→ (B . 3)
This makes it easy to implement things like inheritance or trees of lexical environments.I sometimes wish that Lisp had better ergonomics for hash tables, but normally I am pretty happy that folks don't reach for them first. Frankly, I think positionality is normally good. Can you image how terrible it would be to program a language in which addition is represented as:
add(operand: 1, operand: 2) { Point x 2 y 3 }
Instead of n-ary tuple operation like Lisp, my design limits all functional application/message sends to a form of a receiver object, message, optional it object, and additional map of args in the preceding syntactic way (still not sure if they would have a class or be classless). Multiple dispatch would be limited to just the receiver and the it object.You can see in Smalltalk where this syntax would be helpful in the awkward selector syntax it adopts.
bool ifTrue: [block] ifFalse: [block]
Looks better as bool if {
If
$then block
$else block
}
Where here the $ in slot names means the corresponding value isn't evaluated, a reference to the late John Shutt's Kernel [1] that explored replacing the lambda calculus at the base of Lisp with a lazy vau calculus in order to restore the historically maligned fexpr [2]. Originally they were removed from the language because they made compiling difficult at a time when performance really mattered (1980), and replaced with special forms that weren't first class subjects of the language (and neither really were the syntactic macros that accompanied them). Going back to the original topic, Alan Kay listed Lisp as one of the primary inspirations for Smalltalk, and especially the fexpr, which he saw as underappreciated:> There were not just EXPRs (which evaluated their arguments), but FEXPRs (which did not). My next question was, why on earth call it a functional language? Why not just base everything on FEXPRs and force evaluation on the receiving side when needed? I could never get a good answer, but the question was very helpful when it came time to invent Smalltalk, because this started a line of thought that said "take the hardest and most profound thing you need to do, make it great, and then build every easier thing out of it". That was the promise of LISP and the lure of lambda—needed was a better "hardest and most profound" thing. Objects should be it. [3]
I'm still in search of the "hardest and most profound thing" - here's to hoping it's real.
[0] https://www.codeproject.com/Articles/1186940/Lisps-Mysteriou...
[1] https://web.cs.wpi.edu/%7Ejshutt/kernel.html
I like your syntax, but wouldn't it be better as something like
{ Point :x 2 :y 3 }
or maybe: { Point x: 2 y: 3 }
that little bit of extra punctuation (only needs to be a single character) is really helpful in making clear what is the field and what is the value. Syntactic minimalism is great, but sometimes it is taken a bit too far> Where here the $ in slot names means the corresponding value isn't evaluated
If parameters have type declarations, why not make laziness part of the type instead of part of the name?
I suppose it makes it clear at the invocation site that the parameter is lazy-evaluated. I wonder how important that is though? In Smalltalk, it is very clear, due to [] vs (). In most Lisps, you can't tell from the invocation site whether some symbol is an ordinary defun, or a special form or macro, you have to look up its definition or documentation. In my experience, that isn't a big problem – if you don't remember it, sooner or later you end up having to look it up anyway, to understand what the parameters mean and what it actually does.
On the laziness part I'm torn - quoting is what that amounts to and it never seemed extremely elegant either. The call site disambiguation is nice for knowing whether you can compile something.
I think Clojure has what you're looking for, and the syntax is great, much more light-weight than JSON (the syntax even has its own name: EDN). 'Objects' are first-class hash maps, your example could look like this `{:class FOO :bar BAR :baz BAZ}`.
Here's a circular list for example in Common Lisp:
#1='(foo bar baz #1#)
[Printing that is a good way to overflow your stack]
More usefully, here is an employee who is an acting supervisor of himself:
'(:employee #1="Frank Smith" :supervisor #1#)
There are ways around this problem in JSON but it requires a bespoke parser. In Common Lisp it's built in to the reader.
If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed.
Even if you allow mutable data structures, it is still possible. For mutable compound heap objects (i.e. heap objects which possibly container pointers to other heap objects), they can have a back-pointer pointing to their parent. If you think about it, you can flesh out some rules about how to manipulate these objects which will make cyclic references impossible. When the rules are violated, rather than failing the operation, you could just add a pointer to a deep copy instead. (Not completely deep, you can still share rather than copy non-compound heap objects, such as strings–sharing non-compound objects can't produce cycles.)
Banning direct self-reference doesn't mean nothing can ever refer to itself, just not directly. So, you couldn't have #1='(employee :name "Frank Smith" :supervisor #1#). But you could have something like '(employee :id 1 :name "Frank Smith" :supervisor (ref :to employee :id 1)). At a physical level there is no self-reference, but at a logical level there still is.
Similarly, one would want to allow cyclic references via a symbol, so a symbol can occur in its own binding. So not completely banning self-reference, just limiting it, in the hope that limiting it will limit the issues it causes. I think, using the strategies I've outlined above, reference counting will work as a garbage collection strategy, without the possibility of memory leaks from cyclic references. (Allowing a symbol's binding to refer to itself will not leak memory, since a bound symbol should still exist even if nobody refers to it; to deallocate it, you have to explicitly unbind it, which will remove the binding, breaking the self-reference and allowing the reference count to fall to zero.)
The acyclic heap is a great property for most data but it's annoying for representing programs. It might be reasonable to restrict letrec to creating functions, and ensure they're all statically allocated, which would give an acyclic data heap with direct representation of functions.
It does mean you can't have an anonymous function which recurses, but that's a relatively rare use case. And actually, if you introduce some kind of special syntax which refers to "this lambda", you can even permit direct recursion of anonymous functions. Mutually recursive anonymous functions would be harder to support–maybe not impossible to support, but such an obscure use case it probably isn't worth thinking about.
In order to print this #1=(... #1#) you have to determine that the given object occurs inside itself. In order to print (#1=(whatever) #1#), you have to determine that the given object occurs later in a sibling expression. Knowing that #1# will not occur inside (whatever) doesn't help all that much with anything; it doesn't help you decide whether or not the #1= prefix needs to be printed.
If anything if the restrictions were reversed, it would be simpler: if the only substructure sharing that were allowed was backward references, that helps. Because then we know that the (whatever) object can only occur again inside (whatever), and not after it. To detect nothing but cycles, without caring about other sharing, we just have to maintain an object path across the recursion so that for every visited object we can ask "is this object already in the path from root to here?" If so we have a backreference.
The sibling case is harder. You can have situations like:
((((#1=(foo)))) (((#1#))))
Deep inside the left item in the list there is an object which occurs again deep in the right item. We need to put the objects we are visiting into a hash, which is pretty expensive.> Just by making sure your primitives for constructing values don't permit cycles to be constructed.
But lambda calculus teaches us that if all we have is functions being passed to functions and binding arguments we can build a circular reference. See the middle word of the domain in the browser address bar. So you have to further dumb things down from there if you want to absolutely eliminate cycles in the object graph.
(You can have it so that you don't care about function-induced cycles (only the garbage collector does) by not having any printed notation or other API that traverses the internals of a function. The application can still build a graph using functions, but is "on its own" as far as providing its own functions for operating on it, and serializing it and whatnot.)
Much like the Y combinator comes out somewhat unexpectedly from the pure lambda calculus, my suspicion is that any sufficiently rich data-construction language will allow, intentionally or unintentionally, for some sort of self-reference.
> Extends the code-as-data paradigm to maps and vectors
Basically think of it as a better JSON with non of the issues you brought up.
See: https://github.com/edn-format/edn
And example:
> #myapp/Person {:first "Fred" :last "Mertz"}
Yeah, that really sucks. Objective-S's object literals do. Otherwise they really wouldn't be object literals, would they?
It was really an accident, because Smalltalk's literal arrays use #(), and since I wanted to have dictionary literals use the basic same look, I chose #{}. And also I use {} for blocks and block structure, rather than []. Well, it turns out you can tuck a name between the # and the {}. Yay! Classes. And thus objects!
EDIT:
Just saw that edn uses a very similar mechanism. Neat!
Firstly, anybody who encountered these environments early in their career will always be misty eyed about them.
Secondly, ask them how big the team was they were working in and what the mechanism was they were using to deploy their programs. Smalltalk in particular is particularly difficult to package up into a deployable program that somebody else can run, and collaborating on development is fraught with code sharing issues you just don't get with less exciting environments like .NET or Python or Javascript/Node.
In a modern context, Smalltalk and Lisp make little to no sense, especially when you have available a rich system of code being shared in more prosaic environments. It's a modern miracle that you can build what you need without re-inventing the wheel. There is no better time to be a programmer than now.
Lisp and Smalltalk may well be great for reinventing wheels, for writing un-readable self modifying code and as a challenging intellectual exercise, but all your productivity just went out the window.
Decades ago, Smalltalk implementations would let you dump some part of a system (say, the part that held your application) to a human-readable ASCII file (which is pretty Git-able), and also load that. If you were using it today, while we're using Git, I can't think of a reason a Smalltalk system couldn't have more tight integration with Git.
Regarding Lisps, I'm not sure which school of thought you're coming from, but there are multiple. To someone without firsthand experience, the things you might've heard someone say, like "Foo is great because code and date are the same darn thing", or "Foo is great because it's dynamic up the wazoo", not only aren't necessarily the strengths that others would claim, or they might not mean the same thing.
For example, imagine some kinds of syntactic extension in Lisps, in which some region of syntax isn't a function call. The region is clearly delimited, and it has an identifier right there that can be looked up to tell you the behavior. Someone without experience with that, but who has used an unfortunate kind of "DSL" in some other language, or, worse, has experienced surprise "operator" overloading in a different language, or who's seen unfortunate choices of when to introduce languages and when not, might think this Lisps thing is more of the same thing, when it's not.
I'd say the change log in something like VisualWorks was definitely not git-able in any way that would facilitate merging of several developers code. In fact it was extremely difficult to keep images in sync without something like Envy/Developer.
Lisp to me is like a fascinating fossil.
Doing Git with just that seems only a little different than what we'd do with a collection of source files today in some other language, including branches and merges.
At the same time, one commercial site I worked at was doing its SCM for C code using kludge scripts over SCCS to an NFS volume. The next was putting C and C++ into Apollo DSEE and making SPARCstations jump through hoops to use that (because ancient DSEE on some Apollo DN10ks for SCM and CI/CD was better than the state of the art almost everyone else was using) and my advanced R&D group had just gotten DSEE-descendant Atria ClearCase for greenfield work in C++ from our Sun and HP workstations. The next site was using some dumpster fire of `.BAT` scripts over Perforce(?) for Windows NT-dominated C++ development.
I'd say VisualWorks Smalltalk's wasn't the problem for team development at the time. The state of practice of everything else was more the problem.
Evidently you have some experience with Smalltalk, but what makes you say that about Lisp? Also which Lisp exactly do you have in mind? To me the Lisp family of languages is more like Classical music: it doesn’t satisfy some modern sensibilities, but it has a timeless elegance and will be around long after the latest fad is gone.
The major difference being the ecosystem built around something like C# is miles ahead of anything available for even Clojure. Some people might think that's an advantage, but for people like me whose job is to be pragmatic and "git er dun" it's a major impediment.
Hence, lisp has become a kind of toy from a golden age. I find myself reaching for C# or Python for experiments. I have dabbled in lisp occasionally over the years but never found Clojure very compelling.
It can also have access to the whole Java ecosystem or JavaScript ecosystem. Funny enough, it can also use most Python libraries. And if you're willing to go less official, it also can be used to leverage Erlang's ecosystem and recently Dart and Flutter.
Unless you meant something different than being able to leverage libraries, frameworks and some of the tooling?
Smalltalk is a tool for the creative spirit. It's a very powerful tool for those who have something inside and want to develop it. On the other hand, nowadays, the alternatives that are massively propagated have other characteristics. They are for those who feel better (or calmer) absorbing what someone else has done. They are for those who hide behind the phrase "it is better to ride on the shoulders of others" trying to show wisdom, but in reality revealing dependency and unproductivity.
The problem with Smalltalk is that in computing, the "creative spirit" is a dying breed. Smalltalk is for the creative. Most think that creating is expensive and settle for copying (imitating) or using (claiming to re-use!).
Whoever thinks someone has something to contribute, let him use a Smalltalk, whoever doesn't, let him buy the latest.
Maybe you mean the Smalltalk code that's been "saved" as part of the Smalltalk image snapshot? In which case, why wouldn't we export the Smalltalk code from the image and share those source code files?
So you either try to file out your own changes and merge into somebody elses image, or try to extract theirs for your image.
I realise the state of the art must have moved on since the 1990s but change files back then were a constant source of problems along with how you deploy your image once you are happy with it.
Contrast that with pip or npm. Once I've built my requirements.txt or whatever, I have not only solved how to share my environment with my collaborators, I can use it to build a Docker container and also solve my deployment dependencies.
Smalltalk: play whack-a-mole with change sets until it looks like it's working then strip the image of unwanted stuff and hope it still works. It's caveman stuff.
A few hours ago you told us "Envy/Developer was in heavy use where I worked" so you know that what you describe was not state of the art even back in the 90's.
We ended up going direct to OTI at the time. It about tripled the licensing costs for a smalltalk seat. Due to the costs, it was uncommon in ST shops.
Presumably y'all paid so much more for Envy/Developer because that was state of art even back in the 90's; not what you described.
This assessment seems very unfair to the current environments of both languages. Perhaps you ought to delve in a second time?
I will say though, these languages have their niches where they really shine.
The oft understated power of Lisp is in creating software that is infinitely configurable assuming you know a bit of Lisp. Extensions are a first class citizen in Lisp programs. The configuration is just some lisp code that's compiled into the running program upon loading. The config can even redefine existing functions. One time I used the config file to fix an actual bug in a program that was abandoned by the maintainer. Of course, only programmers want this functionality, and so its usefulness is limited to things like Emacs. But it still has its place.
I don't think this is as big an issue as you think it is. You can find a number of Lisp family language implementations which compile to JVM bytecode, .Net bytecode, or JavaScript – Clojure, ClojureCLR and ClojureScript are the most notable and active (even if Clojure is somewhat of an unorthodox Lisp), but there are others (such as Armed Bear Common Lisp, ABCL). So you can write your Lisp code, and easily consume those massive library ecosystems. Similarly, Smalltalk has PharoJS, and I know there have been ports of Smalltalk to JVM and .Net too (although I don't think they've been as successful).
Technologies for in-process multiple language integration, such as FFI libraries, have greatly improved over the last 10–20 years. Mixing code from different languages is easier than it ever was in the past, and there are further improvements on the horizon (for example, in Java, the planned replacement of JNI with the FFM API).
I think WebAssembly is really exciting – especially if they improve the WebAssembly-JavaScript integration story, which I know they are working on. Compile whatever language you like to WebAssembly, and then you have full access to the JavaScript library ecosystem; and an improved WebAssembly-JavaScript integration will likely also help simplify cross-language integration between different languages targeting WebAssembly.
We have an abundant choice of technologies for inter-process/inter-machine cross-language code integration – everything from relatively modern approaches such as REST, GraphQL, gRPC, Thrift, through to more old-fashioned approaches such as SOAP, CORBA, DCOM, etc. With the rise of micro-services, server-less, etc, it is easier than it ever was to build an application containing a mix of services in different languages. Few care if your app is composed of two or three or four different containers, and even fewer care if the code in those containers is written in different languages.
Smalltalk was (for me) unparalleled at modelling things, including its own self (pun?). I think it also thrived in an era where a high percentage of the programming population pursued it as a form of mastery. The percentage of people that still program with some sort of mastery of the art is much lower now (to wit, I think there are more of them total, but less as a percentage). Working with the stack of internet technologies is like becoming a city planning engineer, a little bit of engineering, a whole lot byzantine rules and policies to navigate, jobs creation program itself in the number of "specialties" that need to be mastered to assemble anything.
I left Smalltalk, because the number of places I could advocate its use was shrinking. Desktop was dying. There's not a lot of technologies for browser delivered programming where going outside of the mainstream is worth the impedance mismatch it causes. And when it comes to handhelds, the hardware vendors have cobbled such an oligarchy of "use our approved/supported technologies" that it's rarely worth it bucking the idiomatic trends. All mainstream code movements live and die on the blessing of iOS, Android, or the browser(s). There's a small remainder of competitive evolution that goes on in the server space, but not a lot. Node may be more modern than Fortran, but half of the basic programs running on the backside are simple enough it doesn't really matter what you write them in, you could write them in Fortran or Basic.
I miss Smalltalk's simplicity. I miss Smalltalk's block closures: sure did make ad hoc functional program easy. I miss Smalltalk's very nicely balanced syntax: kind of code like, but also uncannily natural language at times. I miss Smalltalk's ability to try and see code and code artifacts as something other than text files. I miss Smalltalk's code navigation and execution tools.
Once Smalltalk became a commercial offering, with the need to support and maintain an installed base, it quit changing. Any vestiges of Smalltalk today, are basically Smalltalk-80+. For Smalltalk to "go on", they (we) needed to "burn the disk pack" again.
For me, the ultimate modern language would look something like Smalltalkish syntax and IDE, with Elixir/Erlang like execution semantics, but retaining a strong object/class declaration story (instead of let's pretend dictionaries are structures), and a namespace story somewhat akin to Python (but without the shadow binding behavior), and VM extensibility story similar to Smalltalk\X (unlike other Smalltalks which always ran into a <primitive> when you got to the "performance" parts, Smalltalk\X just supported inline C extensions, so you could just optimize right in place, no JNI/FFI two worlds nonsense). And a world where delivery vendors (browsers, handhelds, etc) had an incentive to simplify program development instead of maintain status quos.
Smalltalk deployment was a PITA compared to compiling a command line tool from C, it was improving, but slowely. Ironically, compared to docker images and microservices and kubernetes and everything else that goes into any "complicated" program to deploy now days, Smalltalk was a breeze.
I have sometimes pondered whether to try making a system that kind of rhymed with Smalltalk, but using Roslyn[0] instead. Although I envisage it as being a kind of playground where you code like Smalltalk, with the artifacts being produced as conventional class files and whatnot so the final build could be done with the normal command line tools.
Possibly. I did the customer side for 15 years at two different companies, and then Cincom (nee ParcPlace). Smalltalk WAS/IS "weird" (er, "pink plane" paradigm shifty, etc) to begin with.
The fact that the total staffs at ParcPlace, IBM Smalltalk team, and Digitalk was smaller than the marketing team alone that Sun allocated for Java, and the insane amount of money spent bootstrapping college kids into training, I'm not sure how the "divide" would mark Smalltalk as different than the other offerings at the time. Sun just did it on a scale easily 2 orders of magnitude greater than Smalltalk.
What team today is bigger than 10 developer?
Where I work dev teams are between 4 to 12 devs at max, most common around 7-8 developers.
So you really don't need a language to support more than that, and if LISP or Smalltalk lets you be faster at that scale it's perfect.
Common Lisp/Scheme on the other hand are pretty bad in that regard, and only iseful for hobby stuff.
Actual Clojure libraries or by wrapping Java libraries?
And because Java interop is pretty good, you can use Java libraries directly from Clojure as well.
I have written REST servers, data stream processing pipelines, hadoop jobs, react-based frontends, mobile apps, etc. in Clojure.
Lisp, Smalltalk, and the Power of Symmetry (2014) - https://news.ycombinator.com/item?id=14333157 - May 2017 (58 comments)
I was hoping you've stumbled on more ideas in the subsequent years. It's surprisingly hard to come up with simple + general.
Ever since you mentioned thread-locals, I've been seeing the pattern everywhere. Our current C++ codebase passes around a "Chip" object (basically, a global configuration holding the current settings for the compiler), and it's remarkable how many lines of code it ends up touching.
And it's not so easy to port your idea to C++ using thread_local. It turns out that Racket thread cells != thread_local storage. The key difference is that when you spawn a new thread, the new thread needs to inherit the current value -- thread_local doesn't automatically give that!
And storing the current "Chip" as a global doesn't quite work either. If we spin up two different compilation threads for two different chips, each thread needs to see its own Chip independently of the other. We sidestep that for now by explicitly passing a Chip to any new threads, but it's not a general solution like your idea was. The moment I realized that, I thought wistfully about your comment from 6 years ago, and I've wanted to bug you ever since for "more like that, please."
I always imagine 3D Lisp-evironment, where you only handle those trees with magical hand gestures like Tom Cruise. But this dream always fails when you go further details, like symbols and numbers: Why would use this archaic system of letters at all? Atoms should be just shapes and sizes.
As a JS programmer by day and a Clojure hobbyist it feels like being a ghost in one of these movies, where they cannot interact with the world directly anymore. It's very frustrating. I sometimes think about all of the unnecessary work that is being done, the proliferation of libraries, tools and "transpilers" just because we lack this basic thing.
I have never played with Smalltalk or some of its dialects or descendants. But judging from articles like this one, I have the feeling that I will end up liking it and then feel even more constrained by mainstream languages.
[1] https://www.quora.com/Why-is-functional-programming-seen-as-...
My understanding is that he'd favor something where the code to interpret the message is also serialized and stored/retrieved and passed around, which is how I think Smalltalk works.
Thus Alan Kay favors objects, which are not actually data, but they are data+interpreter bundled together. Where as Rich Hickey favors data on its own, and facilities to represent and model data alone, independently of a given interpreter, and then he favors having various external things apply their own interpretation of the data.
At least that's my reading of their discussion.
But I don't think these make the larger points any weaker.
I can summarize Smalltalk thing this way: existing Smalltalk implementations (Squeak, Pharo etc.) are not something we should use in real world, they are just proofs of concept and food for thought. Real Smalltalk-like/Object oriented systems were bespoke written for specific machine and specific customer. Current day examples of OO (object oriented) systems really are... the Internet with all of its nodes, operating systems UI windowing systems to some degree (although very crippled and simple), maybe something like kubernetes, but again, it's very crippled and awkward. The ultimate goal is to create an environment, where there can exist objects (let's say processes/agents), which talk to the environment and other objects using messages. On top of all them you should be able to create full programming language which would allow you to compose/iterate/send messages to objects in any way you want, creating as many levels of abstractions as you wish, maybe creating new wrapper objects keeping some other objects inside etc. The programming language itself also should be the subject of real object orientation, for example the debugger or running program should be an object reacting to messages like "stop", "resume" etc. Languages like Rust/Go/JS should be used only to implement internals of some object and react to incoming messages in its narrow area when managing memory, data structures and IO. This is partially true today, we use OS to send messages/pipe sockets etc to the objects/processes. Sending messages should ignore the physical location of the object, just like the Internet does.
Thinking this way, the glue code we write today should disappear in 99%, we should be able to start thinking in higher abstractions, about running processes, computations in time and space, how they relate to each other, make queries to the processes, create interceptors etc. This is that I see when Alan Kay says the computer revolution haven't happened yet and we're still dealing with bag of bad ideas from 60s and 70s. Just packaged in nicer clothes, because we have faster computers and better screens.
We should start thinking longer term and not focus on immediate money as quickly as you see some results, because that's running in circles and real wealth is not created.
This is true of CLOS (the Common Lisp Object System) as well. Every class is also an object. Which itself is an instance of...a metaclass. Which is also an object. Which you can introspect and modify at runtime.
One can get wrapped around the axle by modifying such things casually, but when you really need it, it's quite handy.
> Languages such as Java, C++, and even Python seem to think that “object-oriented” means mostly “classes and inheritance.” Which is sort of like saying that “driving” means mostly “buttons and pedals.”
https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Ob...
One big problem with Java, is the lack of meta-classes. Why can't I subclass Class? There are cases where doing so might be a sensible solution, but Java won't let you. In Java, for my instance methods, I can inherit them from a superclass, or have them implement an interface; but, static class methods cannot be inherited, nor can the static methods of a class implement an interface. This forces people into workarounds such as the factory pattern (which many Java applications/libraries massively overuse), when having a static method implement an interface might be a simpler solution – but that would require us to be able to (effectively) subclass Class. It also produces weird things like 'Class<T>' to mean a class extending class (or implementing interface) T, where meta-classes would offer a much more elegant solution. I don't think the real problem with Java is that it is class-based rather than prototype-based, I think the problem is that its class system is half-baked (no metaclasses, no reified generics, etc)
A point this article raises, which is very true – in Smalltalk, reflection is read-write – modifying classes at runtime is as easy as querying them. Java reflection is essentially read-only. It is possible to do read-write reflection in Java, but it involves immense complexity with source code or byte code generation libraries, custom class loaders, etc – to do something which Smalltalk supports trivially. The argument on the Java side, is that read-write makes optimisations (JIT/etc) a lot harder. No doubt true, but there are alternatives which Java has not pursued – for example, support multiple versions of a class, code is JITted to work with the latest version at JIT time, but other versions are detected and cause a fallback to interpreter mode (possibly followed by re-JITting to add support for that other version.)
This is not how Self works. In Self, it is legitimate to take an existing window, clone it, and change properties and methods to taste. The first window serves as prototype for the second, despite also being a regular window. It's also possible to use what I call the Master Mold pattern (after the Sentinel from Marvel Comics whose only purpose is to make more Sentinels). That is, to construct an object that isn't actually used except as a prototype for other objects. In that case, the prototype will indeed function like a class. But this isn't necessary or even encouraged in Self.
But doesn't this have the risk, that I change some property on the first window, without realising the second window is using the first as its prototype – and while that property change is beneficial for the first window, it is detrimental to the second?
I suppose the same thing can happen in class-based OO – I might make some change to a superclass which is beneficial for one subclass but breaks another. However, I think the class-vs-instance dichotomy helps here – if I'm trying to evaluate whether a change to the superclass might break a subclass, I can focus on the subclasses, and often I can get away with not studying every piece of code which instantiates or uses the class and its subclasses. I can't use that heuristic in prototype-based languages, since the language syntax and semantics don't distinguish unextended uses from extensions.
> It's also possible to use what I call the Master Mold pattern (after the Sentinel from Marvel Comics whose only purpose is to make more Sentinels). That is, to construct an object that isn't actually used except as a prototype for other objects.
Oh sure, but here we are back to "weak-typing" vs "strong-typing". A class-based language enforces the "Master Mold" pattern, and if you try to violate it, your code won't even compile. A prototype-based language doesn't enforce the pattern, it is just in the programmer's head. Being able to break it in an exceptional case might be helpful; on the other hand, there is the risk the programmer might break it by accident rather than intention, producing bugs which would never happen in a more strongly-typed language.
As I recall in Self, the two objects are independent of each other aside from the second being derived from the first upon its creation. Meaning changing one does not affect the other.
"But what if you want to change something about both windows, or all windows derived from a prototype?" you might say. "You would have to change them all manually, for every single affected window!" To which I say the Self developers apparently thought the rigidity of class hierarchy and the fragile base class problem were bigger problems than the unergonomic nature of making sweeping changes across an entire set of derived objects.
Class-based object systems present shortcomings because you cannot accurately predict a class hierarchy that will solve your problem especially if it is unknown. Even in a language as flexible as Smalltalk, class-hierarchy decisions made before a problem was fully characterized impeded the programmers' ability to fit a solution to the problem. OO developers have learned to accept these shortcomings, and the extra work it takes to work around them. Self is an experiment in eliminating these problems, and it just presents a different set of tradeoffs.
For anyone curious, I recommend this talk [0] by David Ungar, one of Self's creators, that explains the history, philosophy, successes, and failures of the Self project. It's a shame more didn't get done with it (though V8, arguably the most important language runtime, is very similar). Someone has even been making a Zig version of Self recently [1].
This is already how it works. You can define any method definition at runtime and the JIT will fallback until it can optmise the new version again. What Java doesn't allow is modifying classes or interfaces because that would compromise the type safety of the language, which AFAIK is not a concern in Smalltalk.
In my head, that's not how prototypes work -- prototypes are objects that serve as a template for other objects. The behavior of the object is defined by its _traits object_ which is pointed to by a parent slot. Therefore there is no difference between modifying the prototype object and its copies - sending the copy message to them will still yield the same result, and one does not affect the other. The only difference is that the prototype object is in a well-known location for copying convenience. Note that this does change once you modify the traits object, but that's not an operation you do lightly anyway.
> Java reflection is essentially read-only.
I believe part of this comes from the bytecode being read-only, no?
> The argument on the Java side, is that read-write makes optimisations (JIT/etc) a lot harder.
Self has proven that it's achievable[0].
[0]: https://raw.githubusercontent.com/russellallen/self/master/d...
This is wrong. C++ wasn't based on Smalltalk. It was inspired by ideas in Simula, which predated Smalltalk. Alan Kay tried to convince people (and succeeded with many) that he owned the term object-oriented, because he was the first person to use it. He wanted it mean only his ideas, but from very early on people were applying it to a variety of ideas that were developing concurrently in the sixties and seventies, his being only one flavor among others. He never gave up trying to convince people that all the other competing ideas were a distorted, mistaken version of his own, but the truth is that those other ideas had their own history and earned their success on their own merits.
> In Simula I, Dahl made two changes to the Algol 60 block: small changes, but changes with far-reaching consequences. First, “a block instance is permitted to outlive its calling statement, and to remain in existence for as long as the program needs to refer to it” [11]. Second, references to those block instances are treated as data, which gives the program a way to refer to them as independent objects.
https://www.sciencedirect.com/science/article/pii/S089054011...
Do you have any citations for this? From the quotes I've seen, it sounds like he regrets using the term 'object-oriented' for the approaches he was thinking of:
http://lists.squeakfoundation.org/pipermail/squeak-dev//2002...
> terms are also "colonized" for political and fad reasons
His version of the story always casts aspersions on alternative OO ideas and frequently, as here, implies devious and mercenary motives to people who were simply working along different lines.
Don't get me wrong, Alan Kay was brilliant, but like many visionaries he had utopian expectations for how his ideas would impact the world. When the future didn't materialize the way he envisioned it, he committed to this narrative that his ideas had been robbed of their success by imposter ideas that stole away attention and energy. It's a shame because there's a much more positive story to be told about the proliferation of different OO ideas that mutually influenced each other and succeeded in different ways.
That said, I don't think classes was much of an improvement. I've seen a lot of profoundly stupid code come out of it, like singletons with a static getInstance(), just because that's what you do in Java, never mind that JavaScript has object literals. Then again before that I saw the same stupid singletons, but implemented with prototypes instead.
This means you can do obvious things like sort an array given a comparison function, or produce a generator from a static array. But it also means you can convert one function into another kind of function. Functional analysis calls these operators. Differentiation is an operator, as is integration, as is the Fourier transform, as is function composition. Many such things don't even need macros, but macros can help.
That is better than cobol, Java …