Thinking Machines – Introduction to Data Parallel Supercomputing (1989) | Hacker News Reader

Thinking Machines – Introduction to Data Parallel Supercomputing (1989)

youtube.com

58 pointsgone3523 comments

I haven't seen this video yet, but I should note that "Thinking Machine's CM2" processor was a 4096x 1-bit SIMD processor.

Which was enough to really have CUDA-like or OpenCL-like code back in the day. Operations could be compiled into 1-bit commands to be executed in parallel across 4096 "SIMD-lanes / threads" (much akin to CUDA-threads).

A lot of the research from the CM2 / Thinking Machines was translated into modern GPU code (parallel prefix sum, radix sort, etc. etc.). The research done back then really lays the foundation upon today's embarassingly parallel works.

There were some other computers that came before CM2 of course. But CM2 + Thinking Machines is very clearly part of the great history of SIMD-compute / PRAM model of compute / Parallel vectors / etc. etc.

Minor clarification -- the CM1 was the 4096-processor machine.

The CM2 had (well, those delivered to customers, anyway) a minimum of 8,192 processors, and up to a maximum 65,535. I worked on one for a number of years. I heard (but never saw) that for internal development at Connection Machines each developer had a single 512-processor board to work on.

You're spot on about the single-bit processor thing though. In their *Lisp implementation, you could declare an integer to be as many bits as you wanted, up to the max 128k bits each processor had.

Is there a *Lisp compiler that generates code to run on current computers?

I don't think so. I'd imagine that stuff was lost to the ages.

Today's compilers that generate a similar set of code are OpenCL, CUDA, DirectX / HLSL, Opengl's GLSL, Apple Metal, AMD HIP, and Intel ISPC.

-----------

Today's computers (or really, GPUs), aren't 4096x wide devices or 65536x wide devices. GPUs are 32-wide or 64-wide natively, and then MIMD'd into parallel parts after that. (AKA: CUDA has the Grid -> Block -> Thread model. I'm pretty sure that Star-Lisp was only Grid -> Thread, with no "intermediate" block in between).

----------

You probably can get a similar effect as the original CM-2 by compiling into AND/OR/XOR and shift-instructions for AVX512 though?

Or maybe AND/OR/XOR + shift instructions on AMD's CDNA2+ processor (64x wide and 64-bit, for 4096-wide SIMD per core). It'd be pretty terrible, all else considered, because modern GPUs have a slight MIMD factor there.

Wouldn't have to generate code for current SIMD units, just being able to compile *Lisp code and run it on a common computer would make me very happy.

In some form it existed: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/...

Lots of other stuff here http://www.softwarepreservation.org/projects/LISP/parallel#C...

There was a variant of GCC (called GCC*, of course) that compiled the C* dialect into "ordinary" C that would run on ordinary computers. You might be able to track that down.

The CM-2 also had hardware floating-point units (off-the-shelf chips from Weitek, paired with groups of 32 bit-serial processors, via custom chips with "transposers", which would transpose 32-bit floats read bit-serially out of 32 bit-processors memories, into 32-bit floats presented in parallel to the FPU chips).

Aka "Slice-wise" mode. The FP units were optional (the project I was on didn't need FP, so we didn't have them). IIRC slice-wise mode was only available via their parallel FORTRAN compiler.

The distinction was slightly more subtle -- "slice-wise" was a storage mode for data in which a 32-bit float would be stored as one bit in the memory of 32 processors, so the "transposers" could be bypassed going in and out. It was supported by the Fortran compiler and by a microcode-like internal layer called "CMIS" (which was, I think, available to at least some customers -- I worked there).

Code in other languages still used the FPUs, if available -- but they paid the "transpose in, transpose out" overhead on every operation.

Thanks for the clarification -- interesting to know!

> I heard (but never saw) that for internal development at Connection Machines each developer had a single 512-processor board to work on.

From at least ~1989 on there were a bunch of machines of various sizes and states of assembly around the building at Thinking Machines. You'd connect to the appropriate front-end and cmattach whatever geometry needed. Certain groups did have dedicated CM-2s, particularly those needing specialized configurations (framebuffer, DataVault, etc).

If you knew where the machine running your code lived, you could go sit late at night and watch the LEDs throbbing. The machine really did have a presence.

Cool, thanks for the update. Yeah, the LEDs were cool -- legendarily good for debugging, but I never heard or saw any way to do that.

1 bit processor and 1 bit commands? is this a typo?

It was a 1-bit processor. But "1 bit commands" is probably a mistake on my part. Perhaps it'd be more accurate to say "commands over 1-bit registers", or something along those lines.

Having spoken to someone who was there (Steve Omohundro who wrote StarLisp), the lack of floating point and extreme parallelism were actually a pain because everything had to be written from scratch. Steve gave me an example that they wrote over 200 algorithms in a new way, including parallel regex (as I recall). So while amazing hardware and ideas, there was a sense that it was along road to purse the actual goal of AI research.

One of their mottos was, "Building a machine that will be proud of us."

I remember reading that as a kid and how much it moved and inspired me.

We’ll get there eventually.

Yeah, same. I still have the scientific American mag issue that they talked about the CM-1 in

I remember a rumor that they originally wanted to call the company "International Thinking Machines" as a dig at IBM, and the motto was going to be "When we say thinking, we mean business."

Anything CM related always reminds me of this Richard Feynman story:

https://longnow.org/essays/richard-feynman-connection-machin...

You can get a CM-1 t-shirt from Tamiko Thiel [1], and a little background [2].

[1] https://mission-base-creations.myspreadshop.com/ [2] https://www.mission-base.com/tamiko/cm/cm-tshirt.html

That is some sexy industrial design.

Reply on news.ycombinator.com