Hacker News Reader

Comment by Peaker | Hacker News Reader

He quotes the guy saying "Computers function by executing a series of instructions".

This is no longer true, and becoming less true all the time.

Using a massive amount of GPU's, and many CPU cores is no longer a simple series of instructions.

Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture.

If we have even more radical changes in hardware architectures, it is almost certain that the more abstract models of programming will survive better than those tied to a particular hardware architecture (Von neumann model).

View on HN

Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture

I have read this many times over the years and it sounds perfect. It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work.

Still, for GPU programming we're mainly stuck with things like OpenCL/CUDA, which are imperative like ever, and it's a lot of work to port non-trivial programs to them.

Also, even with Haskell/Erlang/etc AFAIK you still have to do special handling to get programs to be efficient with manycore parallelism. And that's not even talking about GPU.

Is this system ever implemented or does it just exist in theory?

The basic problem in porting to OpenCL/CUDA is identifying the mapping and reduction operations buried in imperative code. That these data-parallel operations are fundamentally functional is clearly reflected in the primitives that the Thrust and CUDPP libraries provide.

The real challenge emerges during program restructuring and performance tuning. This requires intimate architectural knowledge and frequent trade-offs between conflicting optimisations.

So we're already doing 'fundamentally functional' operations? That's an interesting way to look at it, but it's still driven from an imperative language. The grandparent post talked about functional programming in a functional programming language on a GPU.

Is Theano a step in this direction? ( http://deeplearning.net/software/theano/ )

>> Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture

> It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work.

One of the advantages of age is that you can remember instead of predict.

There are three problems - finding enough work that can be done in parallel, controlling the amount of parallel work that you find (yes, you can find too much), and managing the overhead.

Functional languages help superficially with the first but the latter two are important as well.

One reason why I write "superficially" is that writing in a functional language isn't enough - you have to write programs with the correct dependencies as well. Functional languages merely make it easier for the compiler/run-time to identify the dependencies in the program, which is a very different thing.

Functional programs encode their dependencies as arguments/return-values. In Haskell, using the par combinator is guaranteed to only affect performance, and not the actual result of the program.

This is already a huge boost.

> This is already a huge boost.

Like I said, I've been to this rodeo before. Heck, I even drank the Kool-aid one of the previous times that functional languages were going to enable parallel execution. Since there's nothing new this time, I think that it's safe to say that the result will be the same.

Easy to find and nicely separable dependencies may be necessary, but they aren't anywhere close to sufficient. (BTW - the relevant criteria is not "huge boost" but significant boost, and I don't mean distiguishable from random chance.)

What functional languages did you see before? Were they pure? Could they guarantee that parallelism did not affect the semantics of their programs like Haskell can?

Did you actually see the use of the "par" combinator?

> What functional languages did you see before?

If you're going to argue novelty, shouldn't you be the one describing past work?

> Were they pure? Could they guarantee that parallelism did not affect the semantics of their programs like Haskell can?

Yes. Why is this surprising to you?

> Did you actually see the use of the "par" combinator?

Do you really think that "do something before it is strictly needed in a lazy language" is new?

In other news, your generation didn't invent/discover any sex acts. (Neither did mine.)

It is surprising because pure functional languages have not been around for a long time, especially not ones with good optimizers.

> Do you really think that "do something before it is strictly needed in a lazy language" is new?

That's not what the "par" combinator does.

"par" along with "pseq" is already used, today, to achieve parallelism in Haskell code, and it already is easier to parallelize than in other environments.

Additionally, STM in Haskell is usable in practice, unlike other implementations of software transactional memory, and it also relates to purity (in this case, the ability to distinguish different types of impurity).

All of this work has already been done at the first massively parallel (64k processors in early 1980s) supercomputer startup: Danny Hillis' Thinking Machines. Look at StarLisp, Paralations, and NESL.

Adding lots of GPUs and CPUs doesn't make that not true. Those processing units still work by executing a series of instructions, they just do some fancy things with regards to the order of instructions and communication between units.

It may be more productive to think of things in terms of a more abstract, functional model of computing, but in the end, for anything to actually happen, that abstract model must be translated in to some series of instructions. I think that's what the quote was trying to get at.

It would be quite possible to design a processor that is not designed around series of instructions at all, but something more akin to functional programming. In this case, everything will “actually happen” without any kind of series of instructions.

Maybe my imagination isn't strong enough, but I don't see how a processor could work without being reducible to a series of instructions. The lowest level of "code" you give it might be more "functional", but there's a direct mapping between that and a sequence of states the processor goes through to actually do anything.

In looking for such a thing, I've seen http://en.wikipedia.org/wiki/SECD_machine and http://en.wikipedia.org/wiki/Graph_reduction_machine, but I'm not sure either of these fit your description. Do you have experience with them or another machine you could share?

Reply on news.ycombinator.com