The Rise and Fall of Thinking Machines (1995)
inc.com
inc.com
I work with Ab Initio software sometimes, and its weird and annoyingly "enterprisey", but due to the parallel computing background of the guys who started Ab Initio, the software was capable of scaling up storage and processing long before anyone came up with Hadoop. Obviously its extremely closed-source stuff with a proprietary language used to configure it.
I hate enterprise software companies, but Ab Initio is certainly pretty cool compared to the other ones, and the people I meet there who are old Thinking Machines guys are super brilliant and interesting.
Another interesting point: the author of this article is Gary Taubes, who is the metascience guy who got big into low carb, high fat diets and the science that supports them.
http://www.mission-base.com/tamiko/cm/CM-1_500w.gif
While search I found this awesome post about someone rescuing a panel from a CM-5:
Too small a cell and it runs out of work and spends most of it's time waiting on the network. Too big and you'll run into packaging problems.
It was a SIMD machine, a single instruction at a time was transmitted to all the CPUs (one of the reasons a company had to be formed, the top notch analog EEs he had to hire to make it work had salaries higher than an MIT professor's, which just isn't done), and depending on their state they'd do something, or perhaps ignore it.
How you laid out your data was key; what I remember of the example he gave, each street intersection was allocated to a CPU.
There's of course much better accounts out there, but this might give you a flavor. It's entirely unlike the usual MIMD machines which they and many others built later.
The one I'm reading is for the CM-5, so maybe the earlier arches were different, but to me it looks an awful lot like modern GPU's.
As my leave of absence was ending, I was still involved at Carnegie Mellon and showed up for a seminar given by Danny Hillis [http://www.edge.org/digerati/hillis/] about the CM-1 Connection Machine [http://mission.base.com/tamiko/cm/]. He went on quite enthusiastically about artificial intelligence and neural networks but thought the CM-1 was less suited for numerical programming. Whenever you tell me that you can't make a computer do something, it's a red flag. I believe the Turing hypothesis. The only question is the efficiency of the calculation. I believe in crunching the numbers, doing the calculations, and finding out whether an assertion is true or not.
So while Danny finished the lecture, I scribbled calculations on the back of an envelope. Okay, suppose you did the arithmetic bit serially, like the PDP-8S? A 32-bit floating-point multiply you have to multiply two 24-bit significands, that's going to take 600 time steps. For the additions, a logarithmic number of shifts, each will take 24 time steps...It turned out that a floating-point add would take about 1/2 a millisecond and a multiply a millisecond. Sounds terrible, but you've got 64,000 processors.
DDJ: So you could do 64,000 of these calculations in a millisecond.
GS: That's up in the range of 100 million operations a second, equivalent to a Cray 1, which was the supercomputer of the day. So after the lecture, I went up to Danny and said, "You said this is a terrible numeric machine, but I can tell it's approximately the equivalent of a Cray 1." And he replied, "Can we have dinner tonight?"
[1] http://www.drdobbs.com/jvm/a-conversation-with-guy-steele-jr...
The Connection Machine began as Hillis's Ph.D. thesis
at MIT. "There was a point when Danny was getting
serious about the design," recalls Minsky. "And I said
to him, 'I hope you don't make the ILLIAC IV mistake.'
And he said, 'Oh, what's the ILLIAC IV mistake?' And
I told him."
.
.
.
"About a month later," Minsky recalls, "Danny came
back to me and said, 'Well, I've decided to make the
ILLIAC IV mistake." [1, pp. 203--204]
The ILLIAC IV was a gigantic machine built in the 1970s at the University of Illinois. It had sixty-four processors, each as big as an upright piano because it was built before its time. In fact, they had to use a forklift to plug in the units. It took seven or eight years to build, and the machine was obsolete by the time it was finished. The university gave it to NASA, which promised to use it but had difficulty doing so.
That the project was too ambitious, given the available technology, was not what Minsky meant by the ILLIAC IV mistake. He told Hillis that the concept itself was flawed; it was a mistake, he said, to restrict all the sixty-four processors to doing exactly the same thing at the same time, an expanded electronic version of the Olympic event synchronized swimming. Minsky told Hillis that the processors should be able to operate independently.
"About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake.'" Hillis told Minsky that the problem was not in what the processors did but in how they communicated. He said that intraprocessor signals got stuck in traffic jams because, among other things, the connecting wires were restricted to two dimensions. Hillis recognized that a richer connection scheme was required, all the more so since he wanted to hook up not 64 processors but 65,536. The signal traffic would be equivalent to a telephone network serving 65,536 customers who placed a quarter of a billion calls per second. That was the chief technical problem Hillis faced in designing the Connection Machine.
In the end, Hillis and his co-workers at Thinking Machines settled on a three-dimensional architecture in which the processors were connected as if they formed a sixteen-dimensional cube. What this means is that each processor, although directly connected to only 16 others, was never more than sixteen steps away from any of the other 65,536 processors. Moreover, the possibility of a traffic jam was reduced because in sixteen dimensions there are numerous possible routes between any two processors. [1]
[1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988), pp. 203--4.
It obviously blew my young mind, with the 16 screens, the ability to select an image as the seed for the new series, the endless playability and the vertigo of not being able to ever go back and find any series again.
I don't know if a Cray would have been less or more suited to that kind of interactive installation. But it surely was a incredible feat for the time.
On the other hand he's part of Intellectual Ventures: http://www.intellectualventures.com/inventor-network/senior-...
"We're building a machine that will be proud of us."
What an odd classification.
When I learned about the CM, I thought it was crazy. How could a SIMD machine work for AI? Intelligence emerges from the interaction and integration of multiple, somewhat independent "agents" -- not really the right word but I can't think of a better one. This is Marvin Minsky's own theory, to which I happen to subscribe, but even if I didn't or even if you don't, I still can't understand how Marvin got sucked into this (except that Danny was engaged to his daughter Margaret).
The key point is how SIMD machines handle conditionals. The conditional "branch" is emitted, and those processors on which the test fails simply don't execute the next n instructions. This means that for a complex conditional that's rarely satisfied, only one or two processors out of the tens of thousands the machine might be built with would actually be executing. This is why you wouldn't run a compiler, for example, on a SIMD machine: most of the time, most of the machine would be idle.
Anyway, I wasn't involved in TMC, though I knew people who were. Nobody asked my opinion, nor would they likely have given it much weight if they had (I'm sure there were skeptics with much better credentials than mine who also were not listened to). But I was right. As I once heard someone else comment, the CM turned out to be useful for approximately the complement of the set of problems that it was intended for. It wound up being pretty good for number crunching, at least for a while until better things emerged, but useless for AI.
I guess it depends on how you define AI, from the excellent article linked above:
http://longnow.org/essays/richard-feynman-connection-machine...
"Part Two of Feynman's "Let's Get Organized" campaign was that we should begin a regular seminar series of invited speakers who might have interesting things to do with our machine. Richard's idea was that we should concentrate on people with new applications, because they would be less conservative about what kind of computer they would use. For our first seminar he invited John Hopfield, a friend of his from CalTech, to give us a talk on his scheme for building neural networks. In 1983, studying neural networks was about as fashionable as studying ESP, so some people considered John Hopfield a little bit crazy. Richard was certain he would fit right in at Thinking Machines Corporation.
What Hopfield had invented was a way of constructing an [associative memory], a device for remembering patterns. To use an associative memory, one trains it on a series of patterns, such as pictures of the letters of the alphabet. Later, when the memory is shown a new pattern it is able to recall a similar pattern that it has seen in the past. A new picture of the letter "A" will "remind" the memory of another "A" that it has seen previously. Hopfield had figured out how such a memory could be built from devices that were similar to biological neurons.
Not only did Hopfield's method seem to work, but it seemed to work well on the Connection Machine. Feynman figured out the details of how to use one processor to simulate each of Hopfield's neurons, with the strength of the connections represented as numbers in the processors' memory. Because of the parallel nature of Hopfield's algorithm, all of the processors could be used concurrently with 100\% efficiency, so the Connection Machine would be hundreds of times faster than any conventional computer."
edit: This made me wonder how Moore's greenarray forth computers might be used for neural nets...
But if you wanted to do, say, genetic evolution of your neural net topology -- not sure that had been thought of then, but people do it now -- the CM wouldn't have worked very well.
Anyway, if my rather faint recollection is accurate, the hot application I heard TMC people talking about the most back then was ray tracing.
I wish I'd known the backstory of the antique fire engine that I'd periodically see driving around the Lechmere area (back when Lechmere was still "a thing" :) )
My experience was trying to be the 1st or 2nd person to fill a particular job. I had a great interview process (except for Danny, a friend, initially giving me a hard time for not being in school ($$$)) ... and then nothing. So long, that it was more than a month later, when I'd accepted and started another job, that they contacted me to proceed.
As I understand it (I knew practically everyone who could do this job), it took them three tries to fill it.
The worse story, though, which was well sourced, was that they once contacted an applicant's current workplace to get recommendation info....
In part, I suspect they're another example of "Death by Lethal Reputation" in hiring: http://www.asktheheadhunter.com/halethalrep.htm
Jesus can you imagine AT&T as an AI? Shudder.
If that's not AI I have no idea what is.
(And if you don't think that Hillis envisioned his AI doing things like "answers random questions" and "translating between human languages", what do you think he had in mind?)
An AI program would have some kind of knowledge representation[1] and inference engine that would allow it to reason about boats. If your query was "that thing where a boat gives birth to a tree", it could tell you your query didn't make sense, as opposed to just returning nothing relevant.
By the way, when I tried my nonsensical query on Google, the first result was about a Syrian woman giving birth on a boat. So Google doesn't even understand the basics of grammar, let alone that a boat is an inanimate object that can't give birth to anything. That doesn't sound like "intelligence" to me. There were AI programs in the 1980s that could do better than that.
For example, neural word embeddings learn which words are synonyms without any input (and google recently release a faster algorithm for doing this called word2vec). There are algorithms for parsing sentences, and numerous databases and ontologies representing manually curated 'concepts'. Google could well be using some of these within it's search.
That it doesn't come back and say "that doesn't make sense" might only be because the search is designed to always try to find something, even if it's the wrong thing. That's because it's posed as a ranking problem.
In any case, what is intelligence? One of my favourite sayings on this topic is this (and I forget who to attribute this to, but it's not mine):
"We will never have artificial intelligent. Every so often we come up with a problem to test for intelligence. When a computer finally beats a human at this problem, detractors are quick to backtrack and claim 'well that's not really intelligence' and proceed to come up with a new test for intelligence."
I suspect that this will likely go on for a very long time. In the meantime, we've been using intelligent algorithms for decades already.
Nightmare. Absolute nightmare.
I really, really hope they take on board the feedback they've had, because this is horrible.
I hate to be so negative. Maybe it'll grow on me, although I feel seriously disinclined to make the effort.
Added in edit: I can't even work out how to use the search function to find that discussion. Eurgh.
Added in further edit: OK, now found it, but I'm reluctant to say anything more as my impressions are so unremittingly negative. I hope you guys have thick skins and serious programming ability to make it usable quickly. For now, there's no way I'll be using it, so the search box is now useless. Bookmarking HNSearch.
Final edit: OK, I'm surprised, this has only been submitted on five previous occasions under this title. I thought it was more.
1. sort by date. this is very useful to find discussions which have dropped off the front page but might still have interesting updates.
2. verbatim search (right now "algolia" -> "algorithm")
3. fix the fact that this thread doesn't show up when I search for "algolia" with [last 24h] checked.
2. verbatim search is something we do not provide yet, I opened a feature-request our side :)
3. Have you check [All] or [Comment] instead of default [Story]?
(I've just updated hn.algolia.com with last ranking/design tweaks)
2. cool :)
3. I had checked [All], but the results look much better now (and this discussion shows up).