Go is boring
aeronotix.pl
aeronotix.pl
I've tried without success to dig up an email from Linus Torvalds (on the LKML?) where, responding to an email complaining that a particular implementation of ARM does not breaking any interesting ground, he goes on a mini-tirade about the lack of appreciation for a simple thing done well. I think Go fits the description nicely.
After having spent a great deal of time in recent years doing things like GPGPU and a _whole_ lot of SIMD programming (not to mention a lot of use of the STL, BGL, etc), I have to say I'm less impressed by the boringness (aka taking good, solid choices from existing languages) of Go.
I understand that not everyone is excited about SIMD or generic programming or writing code for 32768 GPGPU threads... but Go feels like a missed opportunity in these respects, solving the problems of the mid-1990s with aplomb (which is good).
It seems more like 'a better Java' - solid, but not genuinely breaking any new ground in a way that creates a single good reason to use it.
SIMD and GPGPU both fairly difficult low-level concepts as they stand: I think there would definitely be some valuable postgrad research in looking at how to create higher-level interfaces to graphics acceleration and GPGPU/SIMD that are as simple and effective as Go's goroutines.
The main problem is that SIMD and GPGPU (even hardware accelerated graphics, to a lesser extent) are bolt-ons; they're not a core part of every computer, which is why they don't usually form a core part of any programming language. At best languages might choose to integrate this functionality into the standard library, but i don't think it will ever come built-in for a general-purpose language (maybe for a domain-specific language?).
I've actually been attempting to write a GPGPU interface between Go and OpenCL[1], so it's certainly not impossible to do GPGPU or SIMD in Go (via cgo), but it would require writing your own 3rd-party libraries or making additions to the go runtime/compiler to develop a novel interface to this functionality like goroutines that can run on the GPU (a dream many people in the Go community share).
APL (and it's descendants J and K) has everything needed to do justice to SIMD/GPGPU. And it had them since the early 60s. I am not aware of an APL compiler that actually uses GPU or SIMD instructions, but one is possible, and wouldn't require any change to the base language.
Not surprisingly, this is because the idiomatic model is closer to the SIMD mindset than to the "standard" (algol/pascal/c/java/python/go) mindset, which is why it is unlikely to ever become popular.
the lack of a magic compiler bullet is infinitely more true as soon as you look at anything remotely like a GPU, which gets into other more complicated problems due to a distinct memory space.
if I were to add any features like that to Go, I'd probably look in the direction of generating ISPC ( http://ispc.github.com/ ) or ISPC-like output. no need to solve the distinct address space issue (which you cannot solve), you have work creation so you don't need to do crazy scheduling hoops like persistent launches on the GPU, and it performs very well on Intel processors for SIMD-friendly applications.
Yeah, and the Athlon64 (and Opteron) for AMD.
Something along the lines of OpenGL shading language, OpenCL C or relevant vector extensions in modern C compilers. With GCC and Clang vector extensions, SIMD programming is fairly nice. (NOTE: GCC 4.8/git version vector extensions emit bad scalar NEON assembly so you can't really use them if you target ARM).
One particular thing about SIMD coding is the shuffling instructions which are hard to express in normal programming language terms as the order of the shuffling has to be static. GCC and Clang both have a shuffling function that requires the parameters to be compile time constants. But I prefer GLSL/OpenCL shuffle syntax, like vec.xxyy or vec.wzyx or vec.s0s0s1s1, and combinations like (vec4)(vec1.xx, vec2.yy). Too bad this syntax is not available on Clang or GCC when doing normal CPU C code.
NEON's shuffling instructions are rather odd to deal with. An odd thing about them is that Clang's ARM intrinsics actually use __builtin_shufflevector to implement NEON shuffles like vrev.
What Go is good for is a different thing. It is good for writing a webserver, which you won't really like to do in SIMD or on a GPGPU. The reason is that a webserver requires you to handle independent activity where things are inherently uncoordinated: while one client is connecting, the other is being sent data.
Vice versa, where Go is bad, is when you want to handle massive parallel computation where each computation is the same and you have to wait until all computation is done before you can continue. This means that you have a sequential program, you have to wait, but not a serial one, since all the computations could be executed on SIMD or a GPGPU in parallel.
Go creates a genuinly unique programming environment. If you come from a C++ background (like me) you might think the C++ solution will always be structurally superior.
But this is not true. It has a really radical take on OOP, actually realizing some of the most extreme takes on OO from the C++ community: no class inheritance, only interface inheritance. Using the inheritance syntax for non-interfaces ("classes") does inheritance by composition.
Regarding templates: C++ has the best (imperative) language support for templates. But it also shows where it can lead... (typedef typename...) Boost has actually become a playground for clean template implementations that look like nothing but a mess.
D's template support is far superior: static if, constraints, compile-time function execution, string mixins, opDispatch. These features make templates much more practical and more powerful.
That's just one example, but anything that involves image processing is a prime example of something that can be optimized for GPGPU.
EDIT: To add another couple of examples:
- Digital effects; rendering, etc. Digital effects studios like Weta Digital are using GPGPU for speeding up their photo-realistic rendering [1].
- Physics simulations for games; games can now move resource-heavy activities like physics simulation from the CPU to the GPU, not only freeing up CPU resources, but also increasing the number of particles, etc. that can be simulated (look for n-body simulations as an example).
[1] http://blogs.nvidia.com/2010/01/nvidia-collaborates-with-wet...
http://webcache.googleusercontent.com/search?q=cache:http://...
I also never seen any language doing interfaces like Go. Go does it just right. I find that this is pretty impressive as a feature on its own.
OCaml has used structural subtyping since the beginning for its object layer. C++'s templates also use structural subtyping on type arguments. Pierce also covers the subject in TAPL.
> I find that this is pretty impressive as a feature on its own.
There are advantages and inconvenients to structural subtyping (compared to nominative): it's more flexible and has much lower overhead, but it's subject to false positives (structurally equivalent but semantically unrelated objects) and tends to have much worse error reporting. Not to mention structurally typed systems still usually have and need nominative types in their core, for "non-object" types.
class A { public: int do() { return 1; } };
class B { public: int do() { return 2; } };
template <typename T> int do(T t) { return t.do(); } do<A>(a)
do<B>(b)
do(Arbitrary)It should work without the template parameter: do(a), do(b)
"Can I take any arbitrary type with an "int do()" method and use that with do?"
Yes.
It's not the case here, so it just works.
Which is why "you have never seen a language doing interfaces like Go", everybody else calls it what it is (structural typing/structural subtyping)
> I no that Go's interfaces are nothing like anything in C++
Well you may "know" it, but your knowledge is wrong.
What did you specialize in? It's pretty easy to hit upon co-routines, if you have an interest in Scheme or even Haskell. But I can imagine, it's much harder if you are into, say, the innards of database schemas. (Just a random example.)
This paper by the authors of Lua has some good historical background:
http://lambda-the-ultimate.org/node/2868
They also poke fun at Python in some ways because the coroutines are fairly limited. Python got them sort of ad hoc with yield in Python 2.4 and send() in Python 2.5. I think Python is by far the most popular language with any kind of coroutine. Coroutines aren't very popular or well understood.
Disclaimer: The above is definitely true for coroutines in general, but I don't know much about Go and whether its coroutines are truly unique.
To add to the list, they're implemented in Python using yield statements
As someone else mentioned, you need to get deeper into programming languages, not operating systems. Scheme has call/cc which can be used to build co-routines. Someone needs to clue in the nodejs crazies as well, callbacks are not the way to design a language. ;-)
Programming advances so slowly because all of the training required to become familiar with it eventually becomes a blind spot.
For more effective use of lightweight processes, I'd probably reach for Erlang or Occam.
In fact, mixing threads and an event loop with async IO is probably a good way to make everything blow up.
GO puts it as a first class citizen in the language. Other languages might call it Actor, light weight thread, fiber, task, etc. It's popular in embedded systems to implement cooperative threading using coroutine due to its simplicity and low overhead in resource utilization.
Have you ever tried to auto complete the 'init' function in RubyMine? It will ask you which one of the 100 init functions do you mean. :)
Not with static typed language. There is only one init function to choose from because the IDE knows the exact type you are working with at all time.
See e.g. DictShield for Python (https://github.com/j2labs/dictshield), Jackson for JVM (http://jackson.codehaus.org/), Swiz for node.js (https://github.com/racker/node-swiz)...
With dynamic languages, you have to hold a lot in your head. Having an enforced structure offloads some things off your brain so you have more mental space to think clearly and not panic or get burned.
class Stream s where
read :: s -> (a, s)
write :: a -> s -> s
instance Stream [Int] where
read s = ...
write a s = ...
instance Stream File where
...
Contrast OCaml, an object type is represented as a set of (method, types) tuples and type-checking is a subset check (if type A has all the methods of type B, then it's a subtype of B regardless of anything else from visibility to semantics):
# let x =
object
method foo = 42
end;;
val x : < foo : int > = <obj>
# let y =
object
method foo = 63
method bar = 12
end;;
val y : < bar : int; foo : int > = <obj>
# x = y;;
Error: This expression has type < bar : int; foo : int >
but an expression was expected of type < foo : int >
The second object type has no method bar
# type simple = < foo : int >;;
type simple = < foo : int >
# (y :> simple) = x;;
- : bool = false
#http://en.wikipedia.org/wiki/Expression_problem
Anyway, there's been quite a few proposals to add extensible records to Haskell, which would allow row polymorphism, similar to what you just showed in OCaml:
http://hackage.haskell.org/trac/ghc/wiki/ExtensibleRecords
Too bad that it has gone nowhere in a long time.
No, I'm comparing structural typing, which is what Go implements, to nominative typing.
But that may very well be due to me having stayed in context of an other sub-thread where this was the subject, and using that as a filter for the current one.
Even if it was, one example of a difficult bug in one system does not invalidate a whole concept.
And duck typing is almost as old as programming itself...
If you want to know how real auto-complete works, try out IntelliJ. Auto-complete on Eclipse is like slowly being pecked to death by ducks.
A more accurate statement in my mind is "Eclipse makes Java palpable, IntelliJ makes Java fun."
I will give Eclipse a try.
[1]: https://github.com/nominolo/scion/
However, there is simply less drive to develop tooling like that for Haskell than there is for Java. Haskell is a much easier language to use given just a moderately intelligent text editor and a REPL than most others. Java, on the other hand, it verbose and annoying even with a very good IDE.
So Haskell can have good support, but since it isn't terribly necessary it isn't anything like a top priority.
http://www.infoworld.com/d/application-development/microsoft...
http://www.sublimetext.com/ https://github.com/DisposaBoy/GoSublime
There is a very real possibility that Go is just not the right fit for you with your current requirements. No language is the right fit for every application.
I'll admit Haskell has a steep learning curve, but once you're there, all this stuff Go is said to do real well feels fairly lackluster compared to what you can find in Haskell-land. What's built into Go can be achieved at the library level in Haskell. As a result, we keep seeing better and better manifestations of key ideas in the library space.
Examples include Cloud Haskell, many, many concurrency libraries, STM, Parallel Haskell, many constant-space data streaming libraries (pipes, conduits, enumerators, ...), several excellent parsing libraries (parsec, attoparsec, trifecta, ...), etc.
As a side note, I used to do lots of python/ruby - I really can't anymore. They feel simultaneously more burdensome to code (no static type-checking), more verbose (no elegant, long pipelines of computations), less expressive (no real first class functions - you barely use map/reduce/fold/etc. in python) and slower (as in runtime).
Having said all that, I do see how Go fills a gap in the market. You need something that's easy to grok and gets just enough of it right that you can produce fast, type-safe-enough and concurrent programs with somewhat less mutable state than what you may be used to in C. A simple mental model and ease of entry are conceivably great for larger, homogeneous teams.
Anyway, I agree that there is no reason why Haskell should be confined to the academic space at all. Actually I think the whole "right tool for the job" mantra is largely misplaced when it comes to Turing complete languages (apart from _very_ low level systems programming).
(meaning both practical and well-written)
There's also a couple of elegant and blazingly fast web frameworks, such as Yesod, Snap and Happstack[2].
2: http://www.haskell.org/haskellwiki/Web/Comparison_of_Happsta... http://stackoverflow.com/questions/5645168/comparing-haskell...
Can you expand on this?
Python has syntactic support for list[0] comprehensions, which can be used a little like maps:
def addOne(n):
return n+1
l = [1, 2, 3]
[ addOne(n) for n in l ] # [2, 3, 4]
def isOdd(n):
return n % 2 == 1
l = [1, 2, 3]
[ n for n in l if isOdd(n) ] # [1, 3]
def accum(s):
acc = s
def a(n):
acc += a
return acc
return a
l = [1, 2, 3]
reduce = accum(0)
[ reduce (n) for n in l ][-1] # 6
There are a couple of problems I've run into. The first is that Python's libraries are just not engineered with the idea of using list comprehensions in this way - folding is as awkward as it looks above, exceptions thrown in the list comprehension functions will terminate the comprehension, many python functions alter state and return None rather than a useful output, and so on. The second is that they're amazingly uncomposable, syntactically:
l.map(addOne).filter(isOdd).reduce(accum(0))
[ reduce(nr) for nr in [ nf for nf in [ addOne(nm) for nm in l ] if isOdd(nf) ] ][-1]
I find Python's comprehension style very convenient, and I'm sure you could produce an excellent theoretical abstraction over it, but if you're coming at it from the point of view of wanting them to be map/reduce or something equally reasonable-about, you're going to be disappointed. Python isn't an object-oriented language, and isn't a functional language - the more I use it the more I think it's something akin to a collection-oriented language. Maybe that's just the way I use it. :)
[0] and also set comprehensions, dict comprehensions, and generator (lazy list) comprehensions, which are wonderful but exacerbate both the problems I talk about.
from operator import add
reduce(add, filter(isOdd, map(addOne, l)))
I'm not trying to be argumentative, I just don't have much experience with that. I write functions that return functions/closures regularly, but they're always simple cases.
By poor support for higher-order functions, I mean that e.g. you have to do "from functools import reduce, partial" for fold or partial function application. It's a trivial complaint, I'll give you, but it's one that's bitten me on more than one occasion (you think I'd learn!). There's also no foldr unless you implement it yourself.
I badly misspoke when I said that you could pickle comprehensions, because what I meant was that the language gives you no hint that you might be able to. Pickling
sum([ os.stat(f).st_size for f in os.listdir(".") ])
[ lambda n: n % 2 == 0 ]
E.g., this doesn't work:
Dump.py:
def isEven(n):
return n % 2 == 0
import pickle
with open('pickled','w') as dumpfile:
pickle.dump(isEven, dumpfile)
Loader.py
import pickle
with open('pickled') as loadfile:
isEven = pickled.load(loadfile)
AttributeError: 'module' object has no attribute 'isEven'
import marshal
marshal.dump(isEven.func_code, file)
#Then to load
isEven = types.FunctionType(marshal.load(file), globals())
import marshal
marshal.dump((lambda n: n % 2 == 0).func_code, file)
#Loading is the same
isEven = types.FunctionType(marshal.load(file), globals())
isEven(4)
Right - that's usually what I want. (I've used pickle to store tests against game assets, e.g. that a model has all of its textures checked into perforce, or that a texture for a model does not exceed 128x128, unless otherwise specified). Marshalling functions is usually a non-starter for this, since a) it's complicated to analyse the call graph before execution, and b) native functions can't be marshalled - an awful lot of code executed against this asset pipeline is thin bindings over native/Java/C# code. Maybe I have found the 1% of the Python use-cases where lambdas suck a bit and everywhere else it's fine--it would be great if my experience was exceptional and no-one else had ever had a similar problem doing something else.
If you wanted to use first-class functions in your code pervasively, then you lack the massive libraries and compiler optimizations available to Haskell. As a result, first-class functions are only used at a superficial level in Python, perhaps as key arguments to some functions.
In a language like Haskell, on the other hand, you make use of the first-class nature of functions all the time.
It's common to have pipelines like:
foldr step 0 . map convert . concatMap (chunks 2) $ inputList
where
step = ...
convert = ...
I've also had a ton of fun mastering and compulsively customizing both VIM and Emacs :-)
If you're not excited about your programming language, I would suggest that maybe you're using the wrong one.
The type system was a bit cumbersome, after coming from Python, especially having to wrangle with pointers after not using them ever, but it's nothing you don't get used to. I'm still not sure how much I gain from static typing, but I'm willing to bear it out.
Channels and goroutines, however, were an absolute dream to use. The entire IRC frontend runs off one process, which, I am led to believe, will basically never need anything more (it just proxies messages from IRC to the backend and back). Communication with the processes was fantastically easy, creating, launching and reasoning about goroutines is, again, very straightforward, and all this feel very much like a first-class citizens.
Python has gevent too, and it suited me very well, but Go feels more integrated and better done.
Note - I am not saying that is the only possible benefit of static typing anywhere, e.g. in Haskell - this is more in line with C, you are doing type declarations to cue the compiler rather than to realize some utopian test-free development methodology.
Of course, this is just from what I hear, I haven't run any benchmarks. Does anyone have more details about this?
Having all this stuff basically solved out of the box makes it very easy to start cranking out actual solutions with Go, even though none of that is particularly interesting from a programming language design standpoint.
https://groups.google.com/forum/?fromgroups#!topic/golang-nu... http://swtch.com/~rsc/regexp/regexp1.html
Edit: missed another good one. There's a lot of discussion about this on the mailing list. https://groups.google.com/forum/?fromgroups#!topic/golang-nu...
Ugh! I can't imagine going back to the days when I had to call a function, check its return value to see whether there was an error or not, then, if no error, proceed to the next function call, check it for error, and so on.
I know exceptions were a source of "complexity", and you could say that Maybe monads or other programming attempts to solve the problem increase complexity too.
But to just punt and go back to doing it the ugly, unmaintainable brute force way? I just find that hard to swallow.
As someone new to Go... What would be the advantages of using Go over Python (taking into account the emergence and future ascendancy of pypy)?
- Go has language-level support, in the form of goroutines, for multithreaded concurrency. Python is single-OS-thread-only, and PyPy doesn't change that.
- Go has enough static typing to help you write safer code, without the verbosity of "bigger" languages like C++ or Java. If you write a lot of tests for your Python app, you might not have variable typos or function argument type mismatches, but in Go the compiler catches these things.
- Go is compiled to machine code. This means that, barring a miracle in JIT/VM research, Go will probably always be faster than PyPy for most tasks.
These, together with the fact that many of the niceties of Python are available in Go (lightweight syntax, first-class functions, iterators and list slicing, etc), make it, in my opinion, a compelling alternative to Python.
Python uses native multi-threads, but the GIL restriction means only one thread can run at a time regardless of number of cores or processors you have.
> If you write a lot of tests for your Python app, you might not have variable typos or function argument type mismatches, but in Go the compiler catches these things.
Use pylint and/or syntastic(for vim).
> Go is compiled to machine code. This means that, barring a miracle in JIT/VM research, Go will probably always be faster than PyPy for most tasks.
Go is slower than Java.
http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
They removed lua-jit from the benchmarks. lua-jit will smoke go in most scenarios. Here is a comparison between lua-jit and lua.
http://luajit.org/performance_x86.html
Native code doesn't mean "always faster than JIT/VM". A good JIT can do optimizations which a static compiler can't. For a long running process with hotspots(same code hit multiple times), a JIT can be as good as(or better) than native code.
<quote>
> Python uses native multi-threads, but the GIL restriction means only one thread can run >> Can run Python code, if you're multithreading for e.g. IO the IO code will generally release the GIL.
</quote>
Yes, native code can release the GIL and run in parallel. As far as it's pure python, only one thread runs at a time. The options are multiprocessing, gevent style concurrency(which I prefer to node's) and native extensions. It isn't as bleak as people make it out to be.
Sorry, that's probably because I got error messages while posting ending up with 2 or 3 comments, and I removed the extraneous ones. You probably tried to reply to one of those I deleted.
Neither statement is really the whole truth, and Python does have some nice ways to do coroutines and futures now, but I see how Go is better here (those 2 features do need to be in the core of the language) and with your other points, thanks.
Actually it's slower or comparable that simple Python in most cases. And slower than Java, which also uses a JIT.
It being "Machine code" doesn't mean much. The implementation also counts, as do the libs. In Python lots of stuff is delegated to plain old C libs.
Part of that is youth, Go went 1.0 just this Spring, and had a serious housecleaning applied in the process. "Being boring" is a big part of Go's identity -- most changes have been pragmatic and thoroughly thought out, and when they come, "gofix" often knows how to apply them to legacy code. (Static linking doesn't hurt, either.)
In theory, Go has more to gain in performance with the GC leaving a lot of room for improvement -- CPython has been subject to a lot of tuning over the years and has found a nice local maximum to settle on. In practice, this does not matter nearly as much as picking the right algorithm, profiling, and avoiding dumb mistakes. (A favorite gaffe: "string concatenation is faster than using a format string in Python".)
Concurrency is a hugh difference. Python's approach to concurrency is the "global interpreter lock" which means only one thread can be running at once to prevent you from trying to access shared objects from multiple threads. Go has lite weight "go routines" that are multiplexed across os threads in parallel and channels that can be used for passing variables between them and the defer statement to help clean up. This sounds complicated at first but it is the simplest system for reasoning about concurrent code I have used.
Go also has pointers, mutexes, fixed sized arrays (with easy to use dynamic slices over them) and other things that let you implement low level data structures when you need to.
Go has interfaces and embedding but no inheritance which leads to iterative development of organization instead of needing to design from the start. The standard library makes extensive use of these and is extremely well organized...I much prefer go's http client code for example.
It doesn't have as extensive a corpus of library implemented in optimized C/fortran and I really miss things like numpy...in many cases python code can be faster because the libraries do all the heavy lifting.
There are tons of other interesting uses for macros. Obviously I'd rather a language had safer equivalents to a C preprocessor.