Startup to Open-Source Parallel CPU

Technically, you don't need to synchronize the cores... it's a MIMD/MPMD system that is not in lockstep. It is up to the programmer (with help from the compiler) to make sure you don't do anything too stupid ;) As for the programming languages, C and Fortran are the big ones to us. We hope once our LLVM backend is improved, you'll be able to run anything you like on the cores themselves. As for the programming model, the three we like the best are CSP (Go), Actor (Erlang), and PGAS (C, Fortran, Chapel, a few other research ones). If you're familiar with SHMEM, that's the closest thing we can think of currently.

When it comes to cost, we're trying to develop as much ourselves to reduce licensing costs. We've been able to do a pretty good job (if I do say so myself) as just two people so far with no capital. Fabrication costs are the killer, with it being about ~$500k per shuttle run, and $5m-$7m for a mask set when we actually go to full production.

Apologies for being critical, I wish you the best.

How much is the scratchpad memory in each of the processors?

>Local scratchpad memories are physically addressed as part of a flat global address space.

So from the programmers perspective each core will have a block of the address space, I.E.: 0-255, 256-511, 512-767, 768-1023 etc.?

Or is there address translation between units? Or if a thread is just built to arbitrarily execute on a unit, it'll have to pre-process its position for name space translation?

Also is there a memory locking in local scratchpad? (I maybe reaching).

Personally, I found that the slide-ins and wobbly text made it difficult to keep my place. I tried Readability, but it didn't do well on it. Scrolling to the bottom to let the glitz run its course, and then scrolling back up to the top to read helped in the end.

How does this compare to other multi-core processors / architectures e.g. epiphany http://www.adapteva.com/introduction/

have you ever heard of Epiphany[1] ? They claim to achieve 70 GFLOPS/WATT. Also the processor seems to be fairly fast and they manage to put 4096 cores on a chip. Though no activity in recent time. Maybe you could find some collaboration points with them..

[1] http://www.adapteva.com/epiphany-multicore-intellectual-prop...

As for comparison to the Phi... it is the fact that our actual core size (and thus the whole chip) is MUCH smaller. The Phi's cores are actually based on the original Pentium architecture (they are just downsized P54Cs) with added AVX instructions. Contrary to popular belief, they are not full x86-64 cores.

Intel has not released the official die size for the Phi, but has said it is about ~5 billion transistors (at a 22nm process), and independent "guesstimates" have pegged the die at 600-700mm2 (there is one place that says 350mm2, but that is false). For the top of the line 61 core Xeon Phi, it uses 300W, with a theoretical peak performance of 1.2TFLOP of double precision performance. That gets you to about 4GFLOP/Watt.

In comparison, our entire will be under 100 million gates, with each core (excluding memory) being around 100k gates. At a 28nm process, our core size (without memory) is a little under 0.1mm2. Our theoretical peak performance per compute chip is 256 GFLOPs double precision, while it should be using around 3 Watts, giving us a performance per watt ratio of ~85GFLOPs/Watt.

Intel has even said that their next generation Xeon Phi, made at their 14nm process, will be at 14 to 16GFLOPs/Watt. At SC14 last week, they made a soft announcement for the following generation at 10nm process will be in the ~2018 timeframe, and that is estimated at only being around 25GFLOPs/Watt.

The bottom line is Intel is just following Moore's law, and is sticking to big and complex systems, which while retaining legacy compatibility, kill you when it comes to efficiency.

The fact that we are using mostly open source tools for our development (such as Chisel: chisel.eecs.berkeley.edu), our development time has decreased and productivity has had a huge boost compared to if we were just writing in Verilog. We also made the decision to not use off the shelf IP, which while difficult to verify, we think we make a much better system. Compare this to the Epiphany implementations, which while having a custom ISA and basic core components, used off the shelf ARM interconnect, off the shelf ARM memory compiler, and many other things that were used to minimize development time and verification. While this is a bit more difficult upfront, since we are keeping our components very simple, our verification is no where near as complex as a "normal" processor. Plus we don't have to pay $500k-$1m+ in upfront licensing fees.

We currently have 128KByte dual-ported SRAM per core (which is physically part of the core, and not a giant array somewhere else on die). It has single cycle latency to the core's registers and to the Network on Chip router.

The on chip mesh network is custom 128 bit wide going core to core. The router can do a read or write to SRAM per cycle AND allows a passthrough to another core in the same cycle.

Our chip-to-chip interconnect is a custom 64 bit (72 lane) parallel interface allowing 48GB/s. There are two of these (unidirectional) interfaces per side, giving you a total of 8 of these interfaces per chip.

Our single precision numbers are actually double that of double precision (compared to a GPU or most other SIMD systems, which have independent FP32 and FP64 FPUs, we have a single combined unit). While our ISA is pure 64 bit, we have packed 32 bit FPU instructions for doing two single precision FLOPs per cycle.