Hacker News Reader

Comment by osteele | Hacker News Reader

That was definitely the intention. It was intended as a systems and application programming language for the Newton.

The original Apple implementation compiled to native ARM code. The runtime was intended to be competitive with C, but by the time we approached that target, large parts of the toolbox had already been written in C, and Walter Smith had created NewtonScript as a scripting language that worked as an alternative for non-performance-critical code. At that point the Cambridge team re-targetted the implementation to build Macintosh applications, but that wasn't a sufficiently compelling (to Apple management) use, and we had lost our executive sponsor when the director of the Apple Cambridge lab was promoted to a position in Cupertino.

For those curious about Dylan's history, the Wikipedia page https://en.wikipedia.org/wiki/History_of_the_Dylan_programmi... looks correct.

(I'm the “Oliver Steele” mentioned on that page. I went to Apple Dylan from Skia – later released as Quickdraw GX – another technology that missed the Newton boat.)

View on HN

I'm the Mikel Evins mentioned on that page.

Everything you say is right. Some details look maybe a little different when viewed from the Newton team in Cupertino.

Yes, as the Dylan runtime got better, the Newton team wrote an OS in C++ with NewtonScript for scripting. There's more back story there, though.

Before any of that work started, an earlier iteration of the OS had already been written in the early Dylan, back when it was still called Ralph. There seemed to be some level of discontent in a few different quarters with that OS. Different people expressed different criticisms (that I heard). Different people assigned blame for their dissatisfactions differently.

It was all pretty good-natured, from what I remember.

But then John Scully told Larry Tesler: Let There Be an OS Written in C++. And there was. There is various gossip about exactly how that came to pass, but it did come to pass.

Larry asked me and a couple of other people to see what we could do with Dylan. We took that ball and ran with it--maybe too far. Like maybe Larry wanted us to run it down to the end zone and instead we ran it across the border and into Patagonia somewhere. We, the five of us or so, wrote a whole OS in Dylan, essentially competing with the 60 or so people working on Scully's mandated C++ OS.

I don't know why we did it exactly, except that we could, and it was really interesting. More to the point, I don't know why Apple management let us keep going with it so long. Morbid curiosity maybe? It did work pretty well, and I think there were some pretty interesting ideas in it.

From a business point of view, though, it was silly. Obviously Apple was never going to ship 2 Newton OSes. Equally obviously, it wasn't going to choose to develop our weirdo Lisp OS instead of Capps' C++ OS that was developed by almost the whole Newton team in response to an order from on high.

The period you describe, when Dylan was getting pretty good and the Capps OS had a lot of features and NewtonScript was pretty well working was well after the inflection point where the main group started working on the C++ OS. Our smaller team started on the second Dylan OS about the same time the other team started on their OS, and we made good (though pointless) headway. We used the same microkernel they did, and the same graphics infrastructure. Everything else in ours was written in Dylan. Dylan worked great.

In fact, that version of Dylan remains my favorite general-purpose programming language ever. I pretty much lost interest in Dylan when it stopped looking and working like a Lisp, but I've never liked anything else as much for day-to-day programming--not even Common Lisp, which is my go-to nowadays.

Creator of NewtonScript here… Hi Mikel! I'm glad to hear you say it was good-natured, because I loved Dylan—having been the Newton team's Cambridge liaison and Ralph cheerleader since the Cambridge team joined Apple. I was just head-down trying to build a small language that had at least some of the goodness of Dylan so we didn't ship something that was just a C++ app in a box!

Hey, Walter! Lovely to hear from you.

I know you loved Dylan, and I know what you were up to with NewtonScript.

I did a little work in NewtonScript for the shipping Newton before I migrated to NeXT. It worked. Or it would have, if Newton had been a better fit for the market of the time.

The Cambridge team (or big chunks of it) later became Clozure Associates, Lisp consultants and maintainers of Clozure Common Lisp. I've worked for them off and on over the years on some interesting stuff.

Well, I ported Skia's memory manager to the Newton, and added some tricks to make it more friendly to the Newton's virtual memory system. So some of Skia made it over ... :_)

The dropping of Dylan was a little more nuanced and maybe somewhat nasty. The decision was made to ship "junior" (the handheld unit that became the MessagePad 100) using C++, while work was still being done in Dylan on "senior", the tablet unit. Eventually the Junior project grew in importance ("Holy crap, we have to ship this in a year") and it was all hands on deck to get it out the door. For a while there were a bunch of Dylan programmers roaming the hallways, they were clutching copies of the C++ Annotated Reference Manual and looking really unhappy. The writing was on the wall, Senior got canceled a few months later and a bunch of the Dylan folks quit.

It's even more nuanced than that.

I was one of the Dylan programmers working on Senior (and Cadillac). I used to have a couple of the hardware prototypes, but I don't anymore.

I don't remember things being nasty, exactly, although I do remember that Larry Tesler authorized the switch to C++ because he was told to, rather than because he wanted to. I remember he looked kind of disappointed. The reason that I and about four other people kept workin in Dylan is that Larry asked us to see what Dylan could be used for on Newton. Our answer was "pretty much everything--see? Here's a working OS."

Realistically, that was the wrong answer, and we should have known it. I think we just got caught up in the really interesting intellectual challenge of inventing a novel OS around a novel Lisp, and nobody stopped us. As I said in the other comment, I don't know why management let us keep going as long as they did, but I'm gratedul.

I might well have been one of the unhappy faces you refer to, but C++ had nothing to do with that. By that point I had been using C++ regularly for about 4 years or so,ever since Apple first got AT&T's CFront, and in those days I still actually liked it.

Admittedly, I liked Lisp a lot better.

But no, C++ was not the source of my unhappiness. My unhappiness was because I had just poured heart and soul into a serious piece of work for about two years and I was having to say goodbye to it.

That's when I went to NeXT. A friend of mine had gone to work for them managing part of the software group and convinced Steve that he should hire me. So I went to NeXt and talked about Newton (because after two years of 100-hour weeks it was all I could talk about anymore) and Steve pitched me his hard sell about how I should be at NeXT.

But that's another story.

> But that's another story.

We'd love to hear it

The development of Newton became famously stressful. I mentioned 100-hour weeks; I wasn't exaggerating. The roughly 60-person main OS team on Newton (that's a ballpark figure; I could be exaggerating or lowballing it) were putting in serious hours to meet ship targets that were gradually solidifying and growing more fraught. Our little team (Matt Maclaurin christened our project "bauhaus") worked maybe even harder because, I think, we knew on some level that our work wasn't likely to be used, and we were trying as hard as we could to beat that fate through sheer effort.

Eventually management got us together and told us that we had met and exceeded every criterion for success, but that it simply didn't make sense to continue developing two OSes. We would need to archive the project and shut it down, and prepare for reassignment to other duties in Newton.

None of us was surprised. We had been aware of the likely outcome for months, and had been trying increasingly long-shot efforts to make the project appealing to someone in Apple. In those days (and perhaps nowadays, too, for all I know), Apple didn't really invent products by having some visionary leader invent a goal and directing engineers to fulfill the vision. Instead, it tolerated skunkworks projects hither and yon, and the visionary leader(s) cherry-picked the ones they thought most promising. Newton itself had been created as a means to keep Steve Sakoman from leaving Apple, rather than as a product vision. So you can perhaps forgive us for imagining that if we just made our project good enough, Apple would find a place for it.

It didn't happen.

As I said, none of us was surprised. We were let down, though, by the sheer intensity of our efforts followed by their sudden end.

I worked on the shipping Newton for a while, writing user-facing code mostly in NewtonScript.

A friend of mine whom I had met at Apple while working on another project had gone to work for NeXT. He was managing part of their software group. We socialized occasionally, and he noticed a change in my demeanor. I explained that Apple had finally shut us down and that I was feeling a little directionless. He said I should come to work for NeXT.

If I remember right, I sort of put him off. I had no ambition at that moment. I was still grieving for our lost project, and for the best programming language I had ever used (it's still my favorite to this day).

He persisted. A little while later I was invited by Steve Jobs to go up to the NeXT offices in Redwood City.

Perry was a big Steve fan. I wasn't particularly, but I did like the NeXT hardware and OS, and Perry knew it. He sold the visit on the basis of the cool hardware and software and getting to see how it worked at NeXT, where all the hardware was NeXT boxes. I bit.

My meeting with Steve was memorable. He carved out a fairly large chunk of time to give me the hard sell about how I should give up on Apple and come change the world with NeXT. I hadn't met him before. His famous charisma was real enough. It was a little odd, though, too. He tried a bunch of different angles to convince me, and he could tell really quickly when it wasn't working. In fact, nothing worked in that meeting. I was skeptical that NeXT would survive, and all of Steve's pitches were pegging my BS meters. It was kind of cool to see him start a pitch, recognize that it wasn't working, and instantly flip to a different one, as if changing channels on a TV. Eventually he gave up and ended the meeting.

I don't remember how long it was. Usually my sense of time is pretty good, but I guess I was too wrapped up in the experience to keep track.

Later (I think it was later) I was invited to be interviewed by several people from their software team. That was kind of weird. Some of those people tried to make it into the kind of hazing ritual that is currently popular in tech interviewing. I performed poorly in that part of it, as I always do, regardless of whether I'm any good for the job. The manager guy running the interview made them stop--I'm guessing because it didn't matter what those results were, Steve wanted to hire me.

I went home and thought it over. They offered me more money than I was getting at Apple. I would get to work on NeXT boxes. I liked NeXT boxes. I had a cube of my own that I had sprung for out of my own pocket. My friend had been right about networked NeXT boxes being cool. You could walk into any ofice on the NeXT campus and log in, and you would get your own familiar desktop environment. That wasn't possible at Apple. I liked Objective-C. I liked Allegro Common Lisp, which came with my NeXT box. I liked Interface Builder--I had learned to like it when it started out as an add-on for Expertelligence Common Lisp.

So, despite my misgivings about Steve's reputation as a jerk, and my mixed feelings about his high-BS sales pitch, and my estimation that NeXT was likely to wither away once Steve ran out of billionaires to soak for "investments", I ultimately decided that the technology was cool enough and NeXT's runway was still long enough that I could go to work for them for a while and have a good experience that would be educational and fun.

And I did.

I didn't stay very long. The thinking was that I would support BSD commands and libraries and several of the apps that shipped with Nextstep for a while, as I decompressed from two years of Newton. Then, when I was feeling perky again, they would let me design a new filesystem. They were intrigued by some of the thinking we did about storage for Newton, and they wanted to see if I could transfer some of it to make a filesystem for NeXT that could be simpler and more accessible for non-techie users while preserving the semantics of UNIX filesystems. That sounded challenging (especially since I was no filesystem expert) and fun.

But, alas, it was not to be. A colleague from the bauhaus project kept bugging me to go to work for his startup. It was a tiny little thing based in Santa Cruz doing contract development of multimedia projects for several clients. It was a bad idea to go to work for him. I had worked for him before and hated it. He was offering me a little more than half what I was getting at NeXT. I lived with my wife and small children in Santa Clara. His office was in Santa Cruz. He promised I wouldn't have to commute, that I could work from home, but he changed his mind soon after I started.

So why did I leave NeXT? One of my colleagues there was a project manager who later told me that Steve blamed her for my leaving, claiming that she had given me stuff to work on that was beneath me. That wasn't it. The truth is that I liked the Santa Cruz guy and wanted to make a collaboration with him work (that was a bad idea), and I was pretty sure NeXT was going to end up petering out, the way bauhaus had. It had nothing to do with the work I was given at NeXT, which I enjoyed tremendously.

In fact, I enjoyed pretty much everything about being at NeXT, except for the constant atmosphere of terror that surrounded Steve.

This thread is why I love HN. Thank you.

I'm not familiar with Newtons' filesystem. What was different about it? I was too young to have one when it was out and the one I was given a few years ago died.

It didn't exactly have a filesystem; it had an object persistence system for "soups". A soup contained "frames."

This Wikipedia describes frames: https://en.wikipedia.org/wiki/Frame_language

Newton's frames were much simpler than full-blown frame systems designed for knowledge representation--just mutable finite maps, really. Basically, soups were persistent graphs of maps, (or dictionaries, or hashes, or objects, depending on your language background). Newton apps were also soups.

I didn't actually work on the shipping soups. I worked on bauhaus, the Lisp-based Newton OS, which had its own frame system that was a little more like what the Wikipedia article describes, though still simplified when compared to a full-blown knowledge representation system.

Apple had one of those, too, by the way. It was called MacFrames, and it was invented by Ruben Kleiman. Both Matt Maclaurin and I had been MacFrames users before getting involved with Newton. MacFrames later evolved into SK8, the Hyoercard-on-steroids project that never made it past management gatekeepers into the wild. I worked on SK8, too, after I returned to Apple post-NeXT, and when I ultimately left Apple it was to go to work for folks that I had met working on SK8.

That's yet another story. Those folks founded a startup named Reactivity, where I worked for another 7 years. Some of them are famous now--for example, one of the founders was John Lilly, formerly of Mozilla and now with Greylock; and another was Mike Schroepfer (everyone called him "Schrep") who is now Facebook's CTO.

I loved Reactivity, too, but I left it because of a health catastrophe. It was later acquired by Cisco, where, coincidentally, I'm doing some work now.

I'm compressing several more tomes worth of stories into the last couple of paragraphs there. I figure I can only ramble so much before it becomes a bore.

They're called "frames" because Bill Luciw used them like KR frames when he built the Intelligent [sic] Assistant. Built-in dictionaries/hashes weren't a common feature of languages at the time, so I had no better idea what to call them. :) I thought the minimum set of data types you'd need for a usable language is primitives, arrays, and records; frames are how you do records.

Right; I remember your discussions with Capps about them in the Living Room.

I agree that built-in finite maps were not a common feature of the languges at the time, and that "frames" were a decent word for them.

After newton I worked on Ruben Kleiman's SK8 for a while (I had used SK8 when it was still called MacFrames for a personal UI-system project, and then later I worked on Matt Maclaurin's GATE, which also used MacFrames). SK8 was more of a full-blown frame system (plus other things; if Newton's Dylan is my all-time favorite working language, SK8 is my all-time favorite IDE)).

The bauhaus frame system was somewhere in-between. It tried to provide a foundation for more frame-system features without actually implementing the more complicated and esoteric ones, like multiple parallel and extensible inheritance systems and truth-maintenance systems and so on. Larry Tesler wrote the initial version of it during his sabbatical, and then I took it over when he, as he put it, "put his executive hat back on."

A couple of “small world” connections:

1. SK8 was implemented in Macintosh Common Lisp (MCL), which was a product of Coral Software in Cambridge, MA. This was one of the justifications for Apple's acquisition of Coral in 1988.

Coral became Apple Cambridge, and went on to create Dylan, which was implemented in MCL.

2. There was a period of time where I had moved to California but continued to work on Dylan. Whenever I returned to MA to work out of the Cambridge office, I stayed with my father-in-law, who had originally proposed “frames”.

Cool!

The small-world connections are dense around Apple in general and around Newton in particular.

Ramble on :)

You should do a podcast. A Debug episode on that would be really interesting.

It's more of a simple database than a file system.

To my surprise, https://manuals.info.apple.com/MANUALS/1000/MA1510/en_US/New... is still up at apple.com. It says:

"Object Storage System

This system is key to the Newton information architecture. The object storage system provides persistent storage for data.

Newton uses a unified data model. This means that all data stored by all applications uses a common format. Data can easily be shared among different applications, with no translation necessary. This allows seamless integration of applications with each other and with system services. Data is stored using a database-like model. Objects are stored as frames, which are like database records. A frame contains named slots, which hold individual pieces of data, like database fields. For example, an address card in the Names application is stored as a frame that contains a slot for each item on the card: name, address, city, state, zip code, phone number, and so on.

Frames are flexible and can represent a wide variety of structures. Slots in a single frame can contain any kind of NewtonScript object, including other frames, and slots can be added or removed from frames dynamically. For a description of NewtonScript objects, refer to The NewtonScript Programming Language.

Groups of related frames are stored in soups, which are like databases. For example, all the address cards used by the Names application are stored in the Names soup, and all the notes on the Notepad are stored in the Notes soup. All the frames stored in a soup need not contain identical slots. For example, some frames representing address cards may contain a phone number slot and others may not.

Soups are automatically indexed, and applications can create additional indexes on slots that will be used as keys to find data items. You retrieve items from a soup by performing a query on the soup. Queries can be based on an index value or can search for a string, and can include additional constraints. A query results in a cursor—an object representing a position in the set of soup entries that satisfy the query. The cursor can be moved back and forth, and can return the current entry.

Soups are stored in physical repositories, called stores. Stores are akin to disk volumes on personal computers. The Newton always has at least one store—the internal store. Additional stores reside on PCMCIA cards.

The object storage system interface seamlessly merges soups that have the same name on internal and external stores in a union soup. This is a virtual soup that provides an interface similar to a real soup. For example, some of the address cards on a Newton may be stored in the internal Names soup and some may be stored in another Names soup on a PCMCIA card. When the card is installed, those names in the card soup are automatically merged with the existing internal names so the user, or an application, need not do any extra work to access those additional names. When the card is removed, the names simply disappear from the card file union soup."

Something I don't get:

"I had worked for him before and hated it. "

" The truth is that I liked the Santa Cruz guy and wanted to make a collaboration with him work "

You meant even though your work collaboration was awful you still liked the guy enough to take a bet (and the promised from-home job) ?

What parts did you enjoy from that language btw ?

Liking someone does not necessarily mean that working with them will make for a good experience. Human relationships are complicated.

I had some experiences working with the guy that were good in some ways and bad in others. The bad parts might have been his fault, or my fault, or both, or might have been because of environmental things that neither of us controlled. There's a good chance that they were caused by a combination of such factors. As always, it's hard to be sure.

He approached me when I was at NeXT. He wanted me to work for his startup. I had reservations. I thought there was a fair chance that it would go badly because of the bad things from before. On the other hand, I still liked him and I wanted my reservations to be unfounded.

They weren't. In fact, it was worse than I feared.

The language I was referring to was the version of Dylan that the bauhaus project used for our OS work. It was basically Scheme plus CLOS plus some functional-programming extensions, and a type system in which all datatypes were CLOS classes.

The development environment, named "Leibniz", was a greatly-extended version of Macintosh Common Lisp that included both the Common Lisp compiler (whose output ran on the Mac 68K-family processors and which was used to implement the whole development environment, including the Dylan compiler) and also the Dylan compiler and runtime, which was initially written for AT&T's Hobbit chips and later ran on ARM.

When bauhaus started our development machines were Mac IIfx boxes with great big Nubus cards sticking out the top. Later they were actual working Newton hardware ribbon-cabled to daughterboards in the Nubus slots.

Leibniz had two sets of everything: one for Common Lisp and one for Dylan. You'd think it would be confusing, but it wasn't, not after the first day or so. There were Common Lisp editor windows and Dylan editor windows. There were Common Lisp Listener windows and Dylan Listener windows. And so on.

That's still my favorite programming language.

I liked the development environment, too, but my favorite development environment ever was SK8, Apple's HyperCard-on-steroids-with-a-built-in-knowledge-representation-system.

If I won the lottery, I'd hire some smart friends to make a new SK8 with an updated version of the old Dylan and a 3D game engine designed for building immersive environments by modifying them as they ran.

I had a talk with B.Mitchener, an open dylan maintainer, he said he liked the old dylan a lot.

The history of lisp is full of forgotten gem it seems.

I hope you hit the lucky number one day

You are right of course. We were involved very early on with Newton development and were asked to start looking at Dylan pretty early on. I still have the original DRM which featured s-expression style syntax.

Soon after though we got an early release of the Newton Tollkit (NTK) which was written in Common Lisp AFAIK.

Maybe that is why I remember tools being heavy weight. The tools, not so much the compiled results.

Reply on news.ycombinator.com