Hacker News Reader

Hacker News

Comment by rbanffy | Hacker News Reader

Parent Full thread

rbanffy

The hot parts we can implement in C. There's not much overhead in crossing over to native code.

View on HN

Lovely theory, but in practice it works more like this:

1) write everything in python because its easy and quick to do so.

2) its slow as.

3) abandon software and write it in something else, or, live on with slow ass software and blame python for being slow and rubbish forever more.

re-writing python in c is a hideously painful process, and its proven to be very unsuccessful practically.

Writing new code in c/c++/whatever and exposing a python api is where successful projects like numpy and tensorflow live.

python is very good at what it is, but no one is ever going to go and rewrite your python code in c to make it faster; its just going to be slow forever.

No, in practice is works something like this:

1) write everything in python

2) yeah, the performance here is good enough so ship it

3) there is no 3

There are very few situations where performance is going to be an issue for you where there is not an existing C module solution that will solve the problem for you. The tired old 'python is slow' trope is getting more and more irrelevant every day. There are other aspects of the language that may make it a mediocre solution to the problem at hand, but out in the real world most people are simply getting the job done with python.

I spent 4 years as a professional python developer.

We certainly shipped (using django) and it was certainly slow, and remains a painfully slow very successful enterprise app.

I’m not arguing that the slowness is deal breaking, but it is slow, and it does, routinely, break the SLAs its supposed to meet.

So... unusably slow? no.

...but slow? yes, it really is.

imo. your milage may vary. /shrug

Unless you are careful, the Django ORM will generate a lot more database accesses than needed. I'd almost bet that most of the time the user spends waiting for the app, the app is waiting for the database.

I'd emphasize that for each tasks that's likely to be a performance bottleneck there are, usually, existing high performance extensions: someone has had the same needs before you.

If there are performance issues, rewriting part of a Python application in C is much less likely than refactoring it, without using other languages, to use an existing high performance library.

Application-specific Python extensions are usually intended to allow scripting of the application, with little concern for Python performance (which is the same as doing the same thing without scripting).

New foreign language Python extensions are usually found in new Python libraries, to make existing proven C or C++ libraries available to Python applications or to improve on existing Python libraries.

Reddit disagrees. And they weren't using 3, which is even slower.

By using Python they were able to ship, which is why you have heard of Reddit and they were able to grow enough to have a concurrency problem (something Python still sucks at); the number of sites of any significance that started by using Java for page delivery is probably somewhere around 0.

Oh cmon, LinkedIn started with Java.

And even after this long it doesn't work quite right.

It's a miracle they shipped it at all.

They would be able to ship in any language, that is what software engineering is all about.

True, but would they be able to ship within the window they could become relevant? Would they be able to add the required features?

You can ship your own clone of Reddit next week, with blazingly fast code, running on two tiny VMs and supporting more load than Reddit, but would it be successful?

Probably, depending on the sales/marketing teams.

Reddit was written in lisp, they switched to python only after having initial success.

Pretty sure Python 3 has had better performance since 3.5 or so?

I still see benchmarks from bilingual projects which show py2 being faster, such as http://falconframework.org/#sectionBenchmarks

YMMV. The benchmark you mentioned used 3.6.

https://hackernoon.com/which-is-the-fastest-version-of-pytho... has a benchmark that includes 3.7.

Nope: Cython + Numba always sufficed so far

So I have to learn Python and C.

With C being an incredibly difficult language to get right.

So difficult that languages like Go and Rust were invented.

Only if the performance of your hot Python parts is a deal breaker, which is almost never.

In almost 20 years, I did it once. And it wasn't really needed, but I wanted to know how much it'd be improved.

Consider Nim language, it compiles to c.

Have you tried Cython already? That's almost Python and much faster.

That is not Python.

It's real-world Python. Why do you think stuff like SciPy are so popular?

Surely not because they are written in real world Python.

Rather real world Fortran, C and C++ with Python glue code.

Just like Tcl, Ruby or whatever scripting language with the same glue code would get out of it.

So real-world Python is:

Two languages to learn. Two languages to support. Two development environments. Two build tools. Two CI/CD pipelines. Two testing infrastructure. Two sets of code and security reviews.

Double the complexity and for what exactly ?

Except when all developers already know C, which is really what you want, seeing how no one ever rises above their level of C proficiency in any other imperative language.

Given the CVE database, all hope is lost if those programmers can never rise about their level of C profency.

Hmm? I'm not saying all C programmers are good, or even that people should prefer C for production code. I'm claiming that if you're bad at C programming, you're not going to be not-bad at programming some other "higher" imperative language.

No C programmer has ever been good. It demands an inhuman level of perfection, or formal methods that boil down to a theorem proving language with C window dressing (the mostly-proprietary stuff Rust competes with).

You'll always need more performance so you'll end up rewriting big parts of the codebase in C because you can't afford the servers and/or can't scale anymore. You'll be kicking the can down the road again and again. Also C is hard. At that point you will realise using Python was a stupid idea when you had other options like Go. It has happened to many people.

You'll always need to integrate libraries already written in C and other languages not under your control. So why use a language that goes out of its way to make that difficult?

That's why Sun's "100% Pure Java TM" campaign and negligent support of integrating Java with native code was so misguided.

Software development is a process. You implement your MVP in one language, then measure its performance with real customers, then incrementally rewrite the hotspots and plug in optimized native libraries as necessary. The language ecosystem and culture should support and encourage that (like C#, Python, TCL and Lua do in spades), not fight against it and condemn you as a mongrel infidel if you dare to miscegenate with other languages.

Sun made a grave mistake with their linguistic supremacist "100% Pure Java TM" propaganda campaign: you should rewrite all of your code in Java instead of expecting Sun to provide you with easy seamless interoperability with existing languages and libraries.

So Sun's JNI and applet web browser integration languished while Microsoft integrated COM and CLR and P/Invoke for seamless interoperability with the web browser and other languages like C++ into Java and CLR. And the insipid influence of "100% Pure Java TM" persists even today (have fun using Android NDK).

Sun also had an unholy obsession with code generation, since their attitude was to breed as much "100% Pure Java TM" Code into the world as possible (see AspectJ, JAXB, etc). While C# took a much better approach of supporting code annotation and metadata and reflection and bytecode generation at runtime so you didn't actually have to generate and compile a huge pile of boilerplate source code (see P/Invoke).

https://docs.microsoft.com/en-us/cpp/dotnet/how-to-call-nati...

http://www.pinvoke.net/

And of course Oracle is delighted to continue pushing "100% Pure Java TM":

https://www.oracle.com/technetwork/java/100percentpurejavaco...

>[...] 100% Pure Java, JavaStar, JavaPureCheck, [...] JavaSpin, HotJava, The Network Is The Computer, and JavaStation are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and certain other countries.

Registering purity as your trademark and insisting that all of your users rewrite every bit of their existing legacy code in your pure language, and never use or talk to any other languages ever again, always seemed like separatist linguistic supremacy to me.

Apparently, C#'s P/Invoke is an evil impure cross-language conspiracy to dilute our linguistic purity and sap our precious bodily fluids, breeding caravans of mongrel coders who will swarm across our open borders and steal our jobs. That's why JNI and NDK are only used by unpatriotic second class citizens with divided loyalties, so they don't deserve to be well supported or maintained.

https://www.zdnet.com/article/100-pure-java-initiative-wins-...

>"Keep Java pure" is the message behind Sun Microsystems' 100% Pure Java initiative which has already won support from 100 developers but will lock out programs such as Visual J++. The move is an attempt on behalf of the Californian firm to maintain integrity of Java code and build awareness.

https://www.cnet.com/news/100-pure-java-watered-down/

>Viewers of cable news network MSNBC may recently have seen a commercial for Sun Microsystems (SUNW) in which a man tells a priest that he's been thinking "pure" thoughts. When the beatific man exits the confessional, he passes a queue of anxious programmers--one wearing a Visual Basic shirt--ready to admit their sins.

>The commercial, which touts Sun's "100 percent pure Java" campaign, is not your usual television ad. But Sun has in many ways shown an almost religious determination to spread the gospel of Java to the far reaches of the globe, even sponsoring a worldwide educational tour for programmers with the support of Netscape Communications, IBM, and Novell.

I don't think I've ever seen a commercial Python codebase that actually did rewrite things in C for speed.

Much more likely is using a library like NumPy where someone else has done that C performance work for you.

Plus in 2019, it's much cheaper to design for horizontal scale and spin up a few more instances.

I have personally rewitten a number of expensive inner loops into C to make Python be less slow. It is easy (if you find it hard Go wont be an option for you). I would say the argument against Python is not raw speed but concurrency model. I use gevent for io bouns studf but going multi CPU with data sharing is hard for the engineer and slow for the code compared to Go and goroutines. Subprocesses are ok and sending data over sockets is ok but serialization several times is sucky. Worse than extra memcpy. Also one big msg can block your interpreter.

Reply on news.ycombinator.com