Hacker News Reader

Comment by wyldfire | Hacker News Reader

Kudos to the pypy team! I suspect pypy would benefit from some kind of marketing campaign. It rarely lets me down and yet lots of python devs don't know about it.

The most popular package(s) that don't work with it have been numpy, scipy, etc. When I first encountered pypy, there was a numpypy and I wouldn't be surprised if that's different now. But bottom line, tons of python code out there needs more performance and has no dependency on numpy or any C extensions.

View on HN

> But bottom line, tons of python code out there needs more performance and has no dependency on numpy or any C extensions.

What they need is nice JIT engines like PyPy and not to switch languages.

If no one invested in improving implementations for modern languages and switched to something lower level all the time, we would still be using Fortran for business applications.

I think you guys are saying the same thing. Ie, because they don't need to switch languages, and PyPy is a JIT engine for Python, these tons of Python code would benefit from if the devs were aware of PyPy.

Indeed that's what I meant and perhaps pjmlp thinks that the feature set in numpy could have been / could be implemented in pure python, now that we have PyPy.

What I mean is that I see PyPy very positive, as a way to stay in Python instead of jumping into C, Go or whatever language.

In terms of performance naysayers only get convinced when someone proves them wrong, not because of what we think might be possible.

So it is important that we have research in GC systems, JIT compilers for dynamic languages, optimizers for FP and LP languages and so on.

Otherwise we might as well keep on using just Assembly.

Numpy is way faster than what you would get if you implementing things in pure Python and executed with PyPy. That's why there is numpypy in the first place.

Devs are already aware of PyPy. If it was as easy as being aware, everyone would switch tomorrow. The problem is that tons of code uses C extensions created with swig or python boost and those take a lot of time to port to PyPy. Another problem is that PyPy is sometimes slower/less memory efficient than CPython when you wrote the code that is optimized for CPython. That is abusing dictionaries and very limited OO use.

PyPy has higher memory overhead for very small units of work but usually once you do more than a couple of seconds of work it usually uses significantly less memory than CPython. As far as I'm concern who cares if a Hello World program uses more memory or some small trivial few lines of code. I'm more interested in real world code which my experience has been that PyPy uses significantly less memory than CPython. If you know of a case where this is not true report it as I'm sure the PyPy devs will consider it a bug.

PyPy also has a slower startup time/overhead so very small amounts of work may end up slower than CPython but in general larger chunks of work will be faster if not its a bug and should be reported.

There is currently some work going on to improve the C extension issue. It's currently in its beginning stages and its goals are to provide both a significant amount of compatibility with existing extensions and a significant reduction in the overhead to call out to C extensions over what is currently implemented in PyPy. On the performance side the new approach was said to remove about 40% of the existing overhead. There are tons of corner cases that still need to be worked on for the new approach to become production worthy and there is no guarantee that there will be no show stoppers that come up or that the tail of corner cases that need to be dealt with becomes a nightmare and the approach is abandoned. If this works succeeds, it will be a game changer as suddenly the majority of libraries would then become compatible with PyPy.

We still are using FORTRAN for Business Applications and I fail to see the need to switch them to a slower language to modernize them. Unless the switch is solving a program leave them alone. This is why in R there is still plenty of FORTRAN code.

Personally I LOVE Python but I always feel it is the 2nd best choice. I do use Python but it rarely is the best tool to use for the problem you are solving. Maybe Pypy will turn this around, and I hope so.

I don't understand numerical analysis as business applications.

For me business applications are what you would use Cobol, Clipper, Java, Delphi, C#, Eifel, ....

Of course Fortran makes sense for number crushing, but that is the language domain, not doing CRUD, ETL or distributed computing stuff (not counting MPI here).

Accounting, spreadsheets, transaction processing, business intelligence?

Of course those are business applications as well.

Do you intend to refer to a product done in those areas Fortran in modern days, maybe with Fortran 2008?

Outside scientific computing and heavy number crushing algorithms and libraries, I don't see a use for it.

No, I'm just saying they're business applications. A segment of business applications are numerically focused. Fortran handles them well. So, in that segment, it can be considered a language for business applications with numeric focus.

I'm not saying anyone should use it today lol. Julia and R are the best for that sort of stuff. Way the hell ahead of Fortran. Although I thought about making a nice web application in Fortran just to screw with people who eventually try to extend it and gasp in horror. ;)

http://www.aim.env.uea.ac.uk/aim/info/fortran.pdf

R != Business Application.

Used R as an example of a language that still uses FORTRAN. BUT I think Microsoft and Business Analysis people would not agree with you.

How can SciPy be fast if it is written in an interpreted language like Python?

Actually, the time-critical loops are usually implemented in C or Fortran. Much of SciPy is a thin layer of code on top of the scientific routines that are freely available at http://www.netlib.org/. Netlib is a huge repository of incredibly valuable and robust scientific algorithms written in C and Fortran. It would be silly to rewrite these algorithms and would take years to debug them. SciPy uses a variety of methods to generate “wrappers” around these algorithms so that they can be used in Python. Some wrappers were generated by hand coding them in C. The rest were generated using either SWIG or f2py. Some of the newer contributions to SciPy are either written entirely or wrapped with Cython.

A second answer is that for difficult problems, a better algorithm can make a tremendous difference in the time it takes to solve a problem. So using scipy’s built-in algorithms may be much faster than a simple algorithm coded in C.

http://www.scipy.org/scipylib/faq.html#how-can-scipy-be-fast...

This is also why the scipy/numpy crowd doesn't care much about PyPy. In this world Python is the equivalent of ducttape. You don't performance optimize ducttape, you want it to be easy to use and to fix any problem you have.

PyPy is taking a similar approach. The difference is Numpy is using CPython's CAPI as the duct tape where as PyPy is using cffi as the duct tape. Both approaches end up using the same underlying libraries.

Too many people just have the wrong impression, thinking that PyPy plans on re-implementing all the libraries that Numpy and scipy use but that's just completely false.

They have re-implemented the Numpy array so that it can take advantage of the JIT and so that parts of an algorithm implemented In Python that uses Numpy can also be optimized. Unlike what occurs when using Numpy under CPython where the Python code does not get optimized unless it is converted to Cython, C Code, or some alternative to Python to have it be optimized.

I like the quote: "Python is generally the second-best language for everything".

we're trying to address both numpy and C extensions, stay tuned ;-)

Keep up the good work!

you should kickstarter this - and also the python 3 support.

we did kickstart the python 3 support on our website. Donate!

Link: http://pypy.org/py3donate.html

Is PyPy going to support Python 3? We kind of need the ecosystem to move forward, and Python 3 has a lot of niceties...

Yes - it's getting there and you can help it get there sooner by donating http://pypy.org/py3donate.html

Doesn't it already? 3.2.5 it seems.

http://pypy.org/download.html

I am a Java dev, I heard of PyPy a few years ago, I find it amazing and often tell people about it when talking about performance and other related topics.

The thing is I've never found another developer that new about it, something is very wrong...

If you are a Java developer and find PyPy amazing, you should also look at Truffle and Graal http://lafo.ssw.uni-linz.ac.at/papers/2013_Onward_OneVMToRul..., http://chrisseaton.com/rubytruffle/, https://bitbucket.org/ssllab/zippy/src/700f88dc2a7d2011258ae...

Also I recently discovered that pypy supports non-ASCII characters pretty well which at least doesn't work with my version of python 2.7.6

In what way?

In variable names, for example. You can do this:

    >>> π = 3.14

In Python 2:

    >>> π = 3.14
      File "<stdin>", line 1
        π = 3.14
        ^
    SyntaxError: invalid syntax

  Python 2.7.10 (850edf14b2c7, Oct 29 2015, 17:32:05)
  [PyPy 4.0.0 with GCC 5.2.0] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>>> π = 3.14
    File "<stdin>", line 1
      π = 3.14
      ^
  SyntaxError: Unknown character

Are you sure you weren't trying it in Python 3 (or PyPy3) rather than PyPy?

I'm getting same result as scardine:

    Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> π = 3.14
      File "<stdin>", line 1
        π = 3.14
        ^
    SyntaxError: invalid syntax
    >>>

also:

    Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> π = 3.14
    >>> print(π)
    3.14

You are right, this is Python 3 stuff. I just assumed the parent was talking about Pypy3.

Reply on news.ycombinator.com