GoPy – Extend CPython by writing Go
gopy.qur.me
gopy.qur.me
Depends, an other category drops down to Cython (which in many cases of low type dynamism consists of just running Python code through Cython).
What are the limitations? Are there situations where GoPy won't work that (eg) ctypes would?
http://docs.python.org/3.3/c-api/init.html
Edit: Even though you can do it in C, Go is a no go. Under item four in the caveats [1] is stated:
| When using goroutines py.Lock must be used. No other
| GIL or threading interface functions have been
| tested - and they most likely will not work.
That's ... interesting.
Of course it would be nicer if the main Go thread didn't lock out the Python caller from running (which it seems to do), but there should still be lots of concurrency available on the Go side.
2. The native way of doing this in C (without using SWIG or cython or any other such)is more verbose.
3. It may be easier (not necessarily faster) to write certain code in Go as opposed to both python or C.
Granted, the vast majority of users use the loadable module approach.
Actually, with OpenMP parellelization is often easy in C and C++. In embarrassingly parellel problems, it is often a matter of adding an OpenMP pragma before the for-loop. To give one example, in some machine learning code I wrote, it was simply a matter of replacing
for (int i = 0; i < static_cast<int>(contexts.size()); ++i)
#pragma omp parallel for
for (int i = 0; i < static_cast<int>(contexts.size()); ++i)
http://github.com/danieldk/par
I think the more convincing argument is that many Python programmers will feel far more comfortable writing Go than C or C++. It offers good performance and some control over memory layout, while being substantially higher level than C and not the enormous language C++ has become.
To be fair, it is trivial to parallelize embarrassingly parallel problems in most languages. Including Python (with `multiprocessing`).
Go separates itself from C/C++ by making concurrent programming much easier. Namely, channels are the predominant form of both communication and synchronization between concurrent processes. This removes much of the need for locks to protect state, which is a source of common bugs.
Go goes an extra step with M:N scheduling, but that is orthogonal to concurrent programming.
Also, you might be interested in my type parametric version of `ParMap` [1]. It loses compile time type safety, but performance still seems to remain roughly the same.
Of course, but usually you are switching to C because the code that is being parallelized needs to be faster.
Go separates itself from C/C++ by making concurrent programming much easier.
Right. I was reacting to the grandparent, who mentioned parallel computation, not concurrency.
Also, you might be interested in my type parametric version of `ParMap` [1].
Nice :). Though my Haskell heart cries when people use interface{} plus reflection as a replacement for parametric polymorphism. Before we know it the library stack will effectively turn Go into a dynamically typed language.
Ah! I missed that. Mea culpa.
> Nice :). Though my Haskell heart cries when people use interface{} plus reflection as a replacement for parametric polymorphism.
As does mine :P But it was a fun exercise nonetheless. If you have a Haskell heart, you might be interested in my write up [1] on writing type parametric functions (using new Go 1.1 features). You will do some bleeding, but presumably you like types, so you might find it interesting. :-)
> Before we know it the library stack will effectively turn Go into a dynamically typed language.
Fortunately, the performance cost of reflection makes practical usage of it very limited. `ParMap` is in the minority here (where the overhead of concurrency shadows the overhead of reflection).
[1] - http://blog.burntsushi.net/type-parametric-functions-golang
import numpy as np
N = 5000
# create a couple NxN matrices with random elements
a = np.random.rand(N, N)
b = np.random.rand(N, N)
# perform matrix multiplication
c = np.dot(a, b)
You can also do a compute-intensive work in parallel using multiple processes:
import random
from multiprocessing import Pool
def fire(nshots, rand=random.random):
return sum(1 for _ in range(nshots) if (rand()**2 + rand()**2) < 1)
def main():
pool = Pool() # use all available CPUs
nshots, nslices = 10**6, 10
nhits = sum(pool.imap(fire, [nshots // nslices] * nslices))
print("pi = {pi:.5}".format(pi=4.0 * nhits / nshots))
if __name__ == '__main__':
main()
For writing new code from scratch, Julia is a much better option. The language is not yet 1.0 and already achieves parity to C compilers in many algorithms.
Go compilers are yet to match JVM and .NET speeds, let alone C.