Only if you're talking about the typical CRUD, website, etc program.
Why should we? People use Python for numerical analysis, scientific computing, etc. Why not let them be able to do that, while still writing in Python (instead of using libraries that delegate to C/Fortran/etc)?
And even a website can do intensive computation work. Why HAVE to write it in 2 languages?
The sad fact is you can't always do this. Then you are forced to drop down to C/C++/Fortran.
It would be great if you didn't always have to do this.
https://www.wakari.io/nb/travis/Using_Numba https://www.wakari.io/nb/travis/numba
And the slides from PyCon Talk: https://speakerdeck.com/pyconslides/numba-a-dynamic-python-c...
For example, consider the problem here:
http://arxiv.org/abs/0903.1817
Ultimately I needed to run a math formula on some surfels and connect an edge in a graph if the formula came out True. Following that I had to prune a bunch of edges in the graph. In 2009 (when I did it), I had to drop down to c++. (Admittedly, this was also partly due to a lack of a good python graph lib.)
I'd be very surprised if Numba is helpful with something like that - the branching and complex data structures don't seem like a great fit for it.
Python and the faster extension languages used are the same kind of tool (well, obviously, both are languages).
So, let's compare them to two hammers.
The difference is Python is made from soft plastic, and while it is light, it only works for softer nails. For harder stuff you need a metallic hammer, which is heavier.
And the sad thing is, the Python-hammer can also be almost as hard as the other hammer while remaining light, it just needs some coating (faster interpreter innards).
So nothing (except besides the man-years of work needed on the interpreter) forbids you to have a hammer both light AND strong (== dynamic and expressive as Python WHILE many times faster).
1) Yes I do (occasionally).
2) Because it applies to everybody, not just me. You surely heard of the plural form of speaking before.
>If not, why are you holding forth on Python's unsuitability for these applications without knowing about the available tools?
I never said Python is "unsuitable for these applications".
On the contrary, I acknowledged that Python IS USED for these applications.
What I said is that the tools you mention (the ones that make it suitable for these applications) are written in other languages, which is kludgy. I'd rather be able to have them written (and modified etc) in Python too.
So, Python + NumPy + etc is suitable for these applications, but Python the interpreted language without external C/Fortran help is not.
The maximum speed of the internet, the maximum speed of your database, the maximum speed of your programming language are all basically constants that you can't improve by throwing money at the problem.
As a programmer, available memory is absolutely your domain or else you're increasing hardware/VM costs for yourself or your employer needlessly. Then you've transferred the cost of increased hardware to host your app/site/whatever off to users.
I can see today bit pinching isn't fashionable as it was long ago when memory wasn't cheap (it's not really "cheap" today either since you're paying proportionally greater per VM than the actual cost), but when a basic tenet like efficiency is ignored because DB all you're doing is compounding the problem.
BTW. Most people already deal with the DB issues with aggressive caching and/or reverse proxy so that still leaves the core application.
In general if your server is making you money, you'll make more money improving the service than reducing the amount of RAM the service takes to run.
In summary, test the conversion rate of a page/feature/etc, not how much RAM it uses. If you're going to performance optimize, attach a profiler and take the easy wins, don't spend too much time on it unless it's a ridiculous amount of resources or your margins on razor thin.
That's not a law of physics. The number of servers you need depends on the number of users you have. The functionality you have to build usually doesn't. So the more users you have, the less true your statement becomes.
I don't think that has anything to do with the languages. I think that has everything to do with the quality of the memory manager implementation, and there is at least one memory manager that does deliver in this respect, namely C4.
There are times when the benefits are greater, for example when your software is running a million instances, or you're working on a hardware-intensive game, but those are certainly not common case.
Once you get past a certain level, though, the cost of the next 1GB isn't $20, it's $20 plus the cost of another computer plus the cost of exploiting multiple machines rather than just running on a single one.
Then it's $20/GB for a while again, then $20 plus the cost of adding another machine, and at a certain point you need to add the cost of dealing with the fact that performance isn't scaling linearly with amount of hardware any more.
That last bit might be a concern only in fairly rare cases. But the first, where you make the transition from needing one machine to get the job done to needing more than one, isn't so rare. And that can be a big, big cost.
(Very similar considerations apply to CPU time, of course. Typically more so.)
This casual disregard for resources would explain why a lot of startups run into infrastructure issues so quickly and settle on terrible solutions (or outright snake oil) to mitigate problems that shouldn't exist in the first place.
People need to start realizing that The Cloud is only a metaphor. Hardware isn't intangible and neither is their cost.
Servers don't grow on trees; they live on a rack and grow on electricity and NAND.
It's easy to think everyone has their acts together like facebook or google, but most companies i've dealt with have hardware upgrade processes measured in months or years, not hours or days. You absolutely have to take responsibility for your work as a programmer and make stuff run fast instead of labeling it somebody else's problem.
Well... For those companies, obviously, dynamic languages are not an option. ;-)
I remember trying to optimize some financial software a couple jobs ago and hitting a brick wall because that's the speed the disk rotates at. We ended up buying an iRAM (battery-backed RAM disk) and sticking the DB on it. You can get this a lot cheaper by avoiding the DB and using a RAM-based architecture if you're willing to sacrifice fault-tolerance under power outages (or if you have some other architectural solution for fault-tolerance, like writing to multiple computers).
I don't understand. Concurrent sessions? Do you mean open connections? Worker threads? What exactly?
If anything, I work hard to put more stuff in RAM to avoid hitting disk, as even though we have SSD's in all our new servers, disk IO is still inevitably where we hit limits first, followed by CPU. Running out of memory is a rare event, and when it is it is consistently an indication of a bug.
For vertical progress, i.e. making complex things possible, these micro-bottlenecks are important. These same DBs with their query processing and other domains with extremely high inherent complexity would greatly benefit from high-level languages yet still the most difficult parts have to be written in the already difficult C(++).
Its just another trade off. Beyond the hype, it has interesting choices, and some very poor ones. (No algebraic datatypes, conservative exploitable will-break-your-program garbage collector, no tail recursion)
But the trade off between performance, safety and compile time is quite unique, and should be considered its main contribution to the language design space. Its just too bad it ignores every other contribution from the last thirty years. Like how to do a type system, or how to write a garbage collection that survives long run proccesses.