Every time I've seen similar done in the past, it has come back to hurt the instigator.
OCaml's also been using tagging to mark unboxed type pretty much from the start, nearly 30 years ago: https://dev.realworldocaml.org/runtime-memory-layout.html#:~....
More modern cpus enforce specific bit patterns for the MSBs, so broken code would be caught immediately.
Well, no, actually; it's:
p = malloc(size+1)+1;
The actual danger is in reinterpreting the pointers, or thereof some arbitrary chunk of memory, to a type that doesn't necessarily satisfy the alignment requirements for that given memory address.
My point though is that this is unlikely to happen by accident. The code above is clearly constructed. It's more work to type it than the correct version. The code makes no sense at all. 1 byte is wasted and it's in general a bad idea to allocate unaligned pointers.
And when you for example provide your own allocator for nodes, you'll probably do it like this:
Treenode treenode_pool[MAX_NODES];
Treenode *treenode_pool = allocate(Treenode, node_count);
https://godbolt.org/z/cjoYvjxGq
<source>: In function 'void foo(int)':
<source>:4:28: warning: pointer of type 'void *' used in arithmetic [-Wpointer-arith]
4 | int * p = malloc(size) + 1;
| ~~~~~~~~~~~~~^~~
<source>:4:28: error: invalid conversion from 'void*' to 'int*' [-fpermissive]
4 | int \* p = malloc(size) + 1;
| ~~~~~~~~~~~~~^~~
| |
| void\*
Compiler returned: 1Second, the error about implicit conversion from void-pointer to typed-pointer is in C++ only -- not in C, where you won't even get a warning. The error happens because you didn't cast the void pointer to the target type and doesn't have to do anything with the fact that gcc _does_ let you (as a GCC extension) do arithmetic on void pointers.
If you remove the "+ 1" you'll still see the same error in C++. In C++, change the target pointer to a void pointer:
void *ptr = malloc(42) + 1;
int *ptr = (int *) (malloc(42) + 1);
I was only disputing the claim that ending up with the unaligned ptr is not trivial for given line of code by providing explanation, and counter-example, why this is not possible. At least not in C++ because that's the language I had in mind.
What you're saying is that it's possible to do void pointer arithmetic in C by using the GNU extension - fine, I can't disagree with that.
Wasn't it part of the reason they ended up with poor "mechanical sympathy" on regular PCs, and got a bad performance reputation as a result?
Of course, it must be said that this performance penalty rarely matters in practice. For most programs, the limiting factor is not performance but the cost of implementation and maintenance. Lisp has has many powerful and convenient features that make implementation faster and surer than in any other language, and which usually greatly lower the cost of maintenance as well. Thus by writing a new program in Lisp you can get it off the ground and earning you money faster than you could in almost any other language. You can also add features more cheaply as time goes on, allowing you to keep up with your competitors. It is only when your system is nearing completion, and all necessary features have been invented and implemented, that you should think about rewriting select parts of that system in a systems language where more efficiency is possible. Not only will you only understand the system after it is written, but you can measure the actual performance of the system to figure out which parts you need to rewrite, and which parts have acceptable performance already and thus do not need to be rewritten.
The limiting factor for what? Their commercial success, or something else?
There's overhead if you mutate old objects in a GCed system due to card marking.
Lisp vectors are vectors of pointers, so there's still an overhead of dereferencing through it. Presumably the objects being pointed to end up compacted together, eventually.
With a compacting GC (ie most of the relevant ones) lists often provide better memory locality than arrays. They only lose at a large scale due to having +8 bytes of memory, which makes arrays occasionally cheaper for large sequences.
Of course not all container accesses require traversal.
One of the reasons why the Lisp machines died out is that round about the mid-80s or so, compiler technology improved for generic 32-bit processors, and it became possible to run Lisp software on a VAX, 68k, or RISC CPU faster than it ran on the Lisp machines' bespoke architecture. Back during the first AI hypecycle, the makers of Golden Common Lisp introduced the Hummingboard to the market, billed as an inexpensive solution to have a "Lisp machine in a PC". It was just a 386 on an ISA card (predating Compaq's 386 PC by about a year) with gobs of memory on board; a special build of Golden Common Lisp allowed code to be run on that CPU rather than the main one.
I'd say Lisp machines died out because Lisp died out (commercially). Other languages got popular, the second AI winter didn't help at all either.
If Lisp itself had fared better (even if it was on generic hardware), Lisp machines could have been saved too, they could still use a VAX, 68k, or RISC CPU underneath, and optimize for the developer experience. Or they'd have turned Lisp machines from hardware into a "Lisp machine" OS or IDE/REPL/etc environment for other OSes succeeding. But none of that took off.
Emacs is that Lisp OS. But it's still lacking a good text editor.
You mean like Allegro CL? LispWorks? Genera on DEC Ultrix?
With the exception of Genera, these solutions are available, maintained, and supported today. Even Genera hung on for a good few years. Lisp machines started to wane a few years before the AI winter hit in full force, because the idea of dedicated hardware to run Lisp programs made sense when your alternative was Maclisp struggling on a PDP-10, but became expensive, slow, and fiddly compared to the generic boxes running fast 32-bit processors with, again, much improved compiler tech. Genera on DEC Alpha, even as an interpreted VM, was so much faster than any of Symbolics's bespoke CPU architectures that Symbolics just quit making hardware and declared the Alpha version of Genera the official upgrade path.
None of these cover my whole description, namely the last part:
"Or they'd have turned Lisp machines from hardware into a "Lisp machine" OS or IDE/REPL/etc environment for other OSes SUCCEEDING".
I wasn't talking about mere existance of those.
My argument was "if the issue was not Lisp falling in adoption in general, but merely Lisp Machines being dedicated hardware (as the parent claimed), then Lisp OSes for generic hardware and Lisp IDEs/envs for popular OSes would have succeeded.
That was a very different business model, very different offering into a different market. One could not move that business model easily, in an existing company to a different market. There were offices, factories, contracts, ... -> costs to get rid of. The thing collapsed, before anything usefully could be scaled down.
For example the Symbolics graphics products were very expensive, just the software. A new owner ported it to SGI and Windows NT machines. To be a technically viable product it used a commercial Lisp vendor. It survived a while in that market (modelling/animation/game tools for game studios, animation studios, ...), changed owners again and then died.
Lisp (the forbidden word during/after the AI Winter) was a part of the problem, but generally a new business model and customers for it wasn't found/searched. For example TI just closed its AI business and never cared about it from then on.
Something like Genera was a huge pile of Lisp code written during 1.5 decades. During its best times the OS and maintenance upgrades were already more expensive than a PC.
Applications from the Lisp Machine were ported away. One no longer needed the OS and no longer had the OS. Some applications died, some survived, some died later.
Some applications (or the development environment) survived for some years on emulators (-> Interlisp-D was ported to SUNs and PCs, Genera was ported to DEC Alpha).
Genera's biggest contemporary problem is that John C. Mallery seems to want to just sit on it rather than make it available to people.
Likely future revenues must be close to nil, so why not open source it? Apparently he talked about it years ago but never actually has.
Yes, a lot of the code is really dated, but if it were open source, maybe some people might freshen some of it up.
From the Lisp FAQ: "CLOE (Common Lisp Operating Environment) is a cross-development environment for IBM PCs (MSDOS) and Symbolics Genera. It includes CLOS, condition error system, generational garbage collection, incremental compilation, code time/space profiling, and a stack-frame debugger. It costs from $625 to $4000 and requires 4-8mn RAM and a 386 processor. "
Later they also ported Genera to DEC Alpha. Currently I have access to an implementation which runs on ARM64 and Intel x64.
There were LISPs for 16 bit MS-DOS. The most famous was arguably XLISP, which was used as the basis for AutoCAD’s AutoLISP extension language, and also the XLISPSTAT statistics package. Another was muLISP, which was resold by Microsoft as Microsoft LISP, and also used as the basis of the muMATH computer algebra system, and also its successor Derive.
They were sufficiently performant to be used in anger. I mentioned some of the real world uses
> or whether large systems could be built with them
I mentioned the muMATH computer algebra system, built with muLISP. I think that would count as a large system, at least by the standards of the time.
On 64bit Intel CPUs this is much less of a problem and Lisp runs roughly twice as fast in 64bit mode. I would think that this even holds without taking advantage of wider data words.
On other architectures this was less of a problem. 32bit CPUs had more registers on a RISC or some other CISC CPUs.
It isn't tagging pointers that makes things (some Lisps are 'things', some are not) slow: it's pervasive abstraction and indirection. Doing some masking on a pointer is pipelined, out-of-order, one to three cycles, and is absolutely dwarfed by cache misses and conditional mispredictions, let alone the sort of pointer chasing which is common in idiomatic Lisp (or indeed Python) code.
Off topic, but that's probably the silliest name for a programming language I've seen. I love it!
I'm pretty sure you can find several more examples in Firefox, as well as other major browsers.
Lua, Lisps, many others.
WasmGC has a i31ref as part of its reference hierarchy, which is implemented with pointer tagging.