Comment by trsohmers | Hacker News Reader

Comment by trsohmers | Hacker News Reader

Parent Full thread

Boy, do I have a new processor architecture for you. For the past 3 years my company (REX Computing), has been working on a new architecture that not only covers both of your points, but removes hardware managed caching entirely and replaces it with software managed (physically addressed) scratchpad memory. By removing all of the unnecessary logic (MMU, TLB, everything required for virtual addressing, prefetching, coherency, etc) associated with the onchip memory system, we can fit more memory on the chip itself, have lower latency to that memory, and use a lot less power. As we fully expose the memory system to our software tools, we can make very good decisions at compile time that replicate most of the features you get in having a hardware managed caching system.

We have silicon back, and are now working with early customers in showing a 10 to 25x energy efficiency improvement for high performance workloads.

I gave a talk at Stanford last week that covers the hardware architecture and shows off our development hardware: https://www.youtube.com/watch?v=ki6jVXZM2XU

Wow. (Not op)

Definitely watching the video later when I have time. Thoughts from my initial impression:

1. Please put benchmarks up on HN when you can :)

2. Would be awesome if you could get this into the hands of low level kernel developers who can tinker with this and see what Linux is like on it - if it's capable of doing that much? (Need to watch the video!)

3. Conservativism FTW (IMO) when thinking about offers [that come in after you've done (1) :P]

1: Around the 30:40 mark in the video I transition to the live hardware demo, where I learned to never do live demos. While it did not work there in person, I do show pictures from it working earlier in the day along with the results from our 1024-point in place FFT assembly test. It was written and tested Monday and Tuesday of last week (finished the night before the presentation), so it is not fully optimized, but we are getting a very good 25 double precision GFLOPs/watt. As a comparison, Intel and NVIDIA's advertised FP64 numbers are around 8 to 12 GFLOPs/watt while they are on a more advanced process node (They are on 14/16nm, while our test silicon was on 28nm).

We're in the process of getting cross platform benchmarks (HPL, HPCG, FFTW, Coremark, etc) up and running, but I'm hoping we'll be posting results shortly.

2. Since Linux 4.2, there has been a mainline port for STM32 which does not have a MMU, so porting linux is technically possible, though not on our priority list. Something like uCLinux would probably be easier, but not as useful. I have no doubt our cores would be able to handle it, it is just that our current customers already expect their software to be running on bare metal.

I didn't watch the talk in full, so forgive me if you addressed this somewhere in the Q&A: What are the differences between your architecture and the Epiphany chips? As far as I can tell they have approximately the same amount of local scratch pad memory and also a 2D mesh routing infrastructure and general overall design philosophy, one major difference seems to be the Serdes design that you have.

Edit: Ah ok, one major difference seems to be that you have quad-issue VLIW as opposed to RISC and apparently you are 10x faster than Epiphany, that is really impressive.

On your edit: RISC and VLIW are not mutually exclusive. I would say we are a RISC architecture (Load/store based/everything happens on registers, fixed instruction word size, shallow pipeline) that happens to have a much simpler instruction decoder & higher instruction level parallelism since our compiler guarantees that the instruction bundle (the 64 bit Very Long Instruction Word containing 4 "syllables" of instructions itself) will only give instructions that will not conflict with anything it is given with or in the pipeline.

Other than we are faster (20% higher clock plus 2x higher IPC) and more efficient (Epiphany is single precision only, where we are twice as efficient as them, we also support FP64), there are three quick points:

1. We actually have double the local scratchpad of Epiphany per core, and our SPM banking and register port scheme actually enables us to operate all four functional units within a core simultaneously, while also having data go in and out of the core on the Network on Chip. With Epiphany, you are very limited in what instructions can run together primarily due to port conflicts... the biggest difficulty is that you can't do any sort of control instructions with anything else.

2. As far as the Network on Chip goes, we are able to guarantee all latencies through a strict deterministic static priority routing scheme. Epiphany had 3 levels of it's Network on Chip, one for stores, one for read requests (8x slower than stores), and one for off chip communications. We have a (patent pending) way of simplifying all of this greatly while reducing latency and having greater bit efficiency.

3. Off chip memory bandwidth is extremely important to us. Even on our test chip, we have 4x higher bandwidth than the Epiphany IV, plus lower latency... our chip to chip and memory interface also uses the exact same protocol as our NoC, simplifying things even further.

There are a handful of smaller things, though my biggest gripe with Epiphany has always been the lack of bandwidth both on and especially off chip. If you are targeting DSP and similar applications like both Epiphany and we are, you really really need to have the ability to saturate your networks and match compute capabilities with it.

How would your Neo ISA compare to the proposed RISC V Cray style vector extension for FFTs?

Our current plans are for very simple SIMD modes that reutilize the same hardware to maximize Area and power efficiency. Right now, we can separately load/store the upper half and lower half of a 64 bit register with 32 bits of data, and while we did not have the time to implement it on this silicon, the plan for the future is to have a mode switch to allow the user to use the same set of instructions/hardware/registers to do double, single, or half precision floating point operations.

The other thing we are looking forward to directly test/compare are unums, specifically the new "type 3" ones known as Posits, which are useful (for some definitions of useful) all the way down to 4 bits, and have a greater dynamic range plus greater precision than IEEE floats while using let's bits and theoretically lower area/power on a chip.

I've had a chance to watch the video, and I have to say I'm really impressed (and really irritated so much of the content is over my head :) hardware design is really fascinating!). I think it's pretty amazing I that I got to learn about this new CPU literally a week after it went public!

I have very little exposure to networked chip designs so I'm not sure how widespread they are, but the Neo architecture reminds me somewhat of the GA144 Forth microcontroller, except "done right." For example, the GA144 lets chips communicate their own stacks (= data) to their immediate neighbors, but you can simply access the memory on another node with the Neo. That allows the worst-case scenario of "I need to access this data and I know it'll be a bit slow" - the compiler can simply work to try to organize memory so that happens as little as possible. The GA144 affords no such option; the other cores need to be instructed to send the data over. Cute design, but much harder to use (even ignoring the fact that the GA144 is Forth).

(As an aside, https://www.infoq.com/presentations/power-144-chip is an interesting (40:12) watch, FWIW. (The video seems finicky so I archived it at https://goo.gl/WGNjuA.) That chip apparently uses 7pJ per instruction. I'm very curious if that's any good nowadays.)

I'm curious if the address space accessible on a given Neo core is "windowed", aka if the address can be remapped. 17:28 says "static routing", but I'm unsure if this affects the memory map. I have the notion that being able to remap would allow for some interesting optimizations, but would add a nontrivial amount of overhead to the chip design.

I think it's awesome you're going to be as open as you can (1:14:40) and release the simulator (1:13:27)!

After thinking about the simulator for a bit, I thought it might be an interesting idea to get it into the education sector for generic "learn RTL" type classes: the hands-down fastest execution/turnaround time makes for a very good pitch and could be an interesting way to get your foot in the door of what I'm presuming is a well-established/entrenched industry, and from there that could be a nice way to springboard into other directions.

Plus, not only does the fastest simulator mean better student engagement, there's also the nice property that everyone who trains on it will collectively groan and weep when they encounter the current "cutting edge" at their first job... best case scenario, you'll wind up with a pile of people who learned on your simulator and want to do real-world stuff at that speed. :)

Idea you've probably thought of but which I'll mention just in case: offering your proprietary optimization stack as an in-house enterprise version for $oh_hey_everyone_can_retire_now, along with a $cheaper subscription compilation-as-a-service version that accepts LLVM bitcode and spits back binaries ready for the "load" command. This would neatly solve many IP issues at once because it's very, very hard to "rip off" an optimizing black box. That said, this makes for some interesting security and trust considerations.

While thinking about all of this I was distinctly reminded of the way POWER8 has propagated. I don't read too much about it on HN, and as Generic Tinkerer #23817 I'm not really too aware of its progress. I think I can safely assume the architecture itself has a laundry list of facepalms; my hope is invested in the fact that it's fairly unencumbered compared to x86, and I hope it goes far. Thing is, it's really hard for me to play with it. I know RunAbove used to have one or two (?), I think either IBM China or a university group in China is offering confusingly restricted public access to some, you can watch the POWER8 nodes in OpenSuSE's build farm. That's about it; there was also the Talos Secure Workstation, but that failed its crowdsourced seed round, which is a huge shame.

My point is that, the simulator is likely going to suffice for most people's interests and purposes, but for the few that want >1MHz (or ±300kHz on the kinds of machines that are likeliest to be widespread in generic education), well, it would be kind of cool for this not to wind up as yet another thing that only hotshots in big industries get to play with, because this thing runs at 1GHz already and looks really interesting.

I envision a possible solution as a simple, fast pipeline that accepts LLVM bitcode (or source code - that works too), compiles it, sends it to a dev board (sitting in a pool of of 3 or 4), and returns the result. Besides queue waits and actual CPU runtime, I can't see the extra steps taking too long, and in ideal circumstances users could probably do several iterations per minute. 4 or 5 cards could very probably fit in a 2U enclosure with 1U more for an x86 blade to run a web server. I have no idea if this sort of thing would be appropriate for this architecture, I'm only thinking about general exposure, educational access, etc - I'd expect that the industry has well-established communications channels and that most of the people that are likely to be the most interested in this architecture either know about it or will know very soon.

Regarding running Linux, I totally get the focus on bare metal DSP-type applications, but I do wonder if there are any potential performance gains to be had from a compiler and kernel (Linux or another) designed to understand software-defined memory management. I wouldn't be surprised if Linux wasn't ideal as a general-purpose OS; besides not really being great at hard realtime, I suspect an "SDMMU" would be difficult to elegantly wedge into the memory management because of architectural assumptions made by all of the code that might thwart the optimizations that could otherwise be afforded.

Finally - after hearing the bit about the CDC 6800 in 1:18:21, I thought I'd mention this just in case (might be boring/irrelevant): a few months ago I stumbled on a (real) PDP-11/70 running RSX-11M-PLUS and TCP/IP, with an open guest account that anybody can use. I mention this in case you're curious to study its I/O characteristics or whatnot (since it's a real machine). You could probably do pretty much any kind of test you wanted on it - I've found that its owners have no issues with rebooting/fixing crashes. Very happy to share details (to anybody!); my email is in my profile, I fear the machine would get trampled if I mention it publicly (on here) :)

(And shoutout to using Nedit. Haven't seen that in a while!)

Thanks for your reply, and thanks for reading :)

This is so cool to hear. I've long been holding out hope that we won't be trapped in the current CPU paradigm forever. Thank you for the work you're doing.

Reply on news.ycombinator.com