Hopfield Networks in Go | Hacker News Reader

Hopfield Networks in Go

mlexplore.org

139 pointsstehat19 comments

I encountered Hopfield networks in LLVM [0]. I still do not understand how you would model Spill Code Placement as a Hopfield network.

[0] https://github.com/alexjordan/LLVM-TMS320C64X/blob/master/li...

The number of neurons Hopfield networks can use seems like a serious limitation. I think the max number in his graphs was 300? Does anyone know if this is a real problem or not?

On another note, what is the state of the art in "'memory networks". I'm thinking beyond LSTMs to a network that might be able to remember high dimensional features across a large span of time.

Usually if you properly use GPU the real limit is on the order of 20k. I think the graph in his post is old as crap, actually

(if you sparsify connections and do some tricks with sparsification sky's the limit, O(n * small multiplier over 1) with memory)

Hopfield nets don't do memory like LSTMs do. LSTM's already sparsify features and memorize them a long time. There's a medium size literature on adding more shit (NTM, Pointer net, attention) but nearly all of it is not that good (it gets tested on bullshit)

Golang: a language where you divide statements with `if err != nil`

A language where it's natural to think about and properly handle every error condition.

Someone who gets it.

In Go, this is one of the most difficult concepts of the language to wrap one's head around. Moreso, it's a concept that makes you bang your head against the wall at first; at least it did for me. After being so used to "getting away with" not thinking about this in other languages, being forced to think this way in Go can be maddening.

Then a light goes on.

...and then you look at your old code and go "eeesh"; in my case, I was coming from a PHP background at the time I was learning Go.

Don't get me wrong - I still like PHP and other languages, and I don't use Go much as it is, but once I wrapped my head around many of it's "strangeness" (compared to other languages), I really appreciated it (because I recognised that the whole point of the language was to help prevent common errors and in many cases force better programming practices to that end), and began to enjoy it.

I hate to be the one who brings Rust into a Go thread but if you find forced error handling to be a revelation then you'll find the borrow checker does the same thing for your program structure and architecture.

There's also a ton of really nice error handlers[1] that let you ergonomically deal with errors while still being forced to confront them.

[1] - https://doc.rust-lang.org/book/error-handling.html

> I hate to be the one who brings Rust into a Go thread

You shouldn't feel bad for this. The Go programmers are legitimately unaware that there's a better world out there.

Honestly, not at this point of HN. I don't think there was one thread that mentioned (or didn't mention) anything about Go which didn't have comments criticizing it.

Everyone knows its problems, and everyone knows about Rust.

It's just that some people don't need such extreme Real Time that they can't afford GC at the expense of the borrow-checker and lifetimes.

> Everyone knows its problems, and everyone knows about Rust.

I guess they've at least heard of it, yeah, but how many of them have actually tried Rust (or any other language with a modern type system)?

> It's just that some people don't need such extreme Real Time that they can't afford GC at the expense of the borrow-checker and lifetimes.

They should probably be using Java, C#, Scala, ML, F#, Haskell, etc. then.

I have tried Rust, Go, Java, SML, Lisp, Javascript, and many other languages. I prefer Go in many situations where simplicity, speed of compilation, and portability are important. Rust is an interesting language with a lot of potential but the jury is still out on how wide the adoption will be. There is a steep learning curve, and the borrow checker has limitations.

One thing I don't appreciate is being lectured by pimply-faced nerds about how Go programmers should embrace Java/C++ generics into their heart as their personal saviors. I understand what Java and C++ are, I can choose to use them or not, and it's none of your business.

Gee, it's almost like Rust and Go address different problem spaces ...

Hmmm... perhaps Rust programmers are legitimately unaware that other people's problems differ from their own?

> In Go, this is one of the most difficult concepts of the language to wrap one's head around. Moreso, it's a concept that makes you bang your head against the wall at first; at least it did for me.

You imply you agree with the parent poster, but if it was "natural" to do error handling this way, surely it wouldn't be the language's most difficult concept?

It is natural imo. I think the reason people struggle with it is because of how different it is from most modern programming languages.

Which is a great idea: knowing which expressions can fail and which ones cannot. However, the way go encodes this idea isn't very ergonomic or elegant. I'd suggest that you take a look at ways other languages do this. For example, the `Either` type in Haskell or the disjunction type `scalaz.\/` in Scala.

Personally the syntax seems like a regression to me, as a JavaScript programmer who moved on to promises from the old callback function(err, Val){ if err pattern. Higher order control flow abstractions are really nice, but hard to do in go because of the weak type system. If it was fully dynamic we couldn't worry about it, and if it allowed a bit more power like generics we could have elegant stuff like Haskell's either monad.

Feels like Javascript callbacks with the error first arguments.

Not something I miss, since promises became more available.

Much more readable than using `;` as a separator, imo.

Go is bad but encouraging thorough error handling is one of its few strengths.

Reply on news.ycombinator.com