Could you tell me what (and how long) your experience is?
My answers about your complaints are basically:
1. Solving threading is (relatively) easy, if you just give up backwards compatibility; Not the 2.x -> 3.x compatibility which is comparatively trivial - but in a major way, breaking every single extension library, and the vast majority of Python code (or slowing it down unbearably). You might be happy, but the rest of the Python users (basically, the reason you actually use Python) won't.
2. Assuming they agree with you about their failure (I dont, FWIW), someone has to step up and offer an alternative. Who is that, what are they doing these days, and why do you think they will succeed where Guido et al "failed"?
3. Perl at 28 years is not much older than Python at 24 years (relatively speaking), but has been sliding into obscurity for a long time now, whereas Python is flourishing. The Python 3 transition is actually happening as planned (IIRC, there was an expected 5 year period just for feature and speed parity!). Perl 6 is esoteric, Perl 5 is aging and dying. I think that for a living, popular language, the Python team is doing a commendable job, even if they do not address a specific issue (that many people care about, but would actually not make that much of a difference in practice if we are to learn from other languages)
It just need further dev work and acceptance into py3
Definitely not "solved without breaking backwards compat".
Before answering your questions 1 thru 3, some context. It is my strong belief that the authorities are constantly looking at golang and JS as their competitors, in other words, the web world, whereas the real hardcore advantage of Python is in science and numerical computing. As evidence, witness numerous Python books which advise new users to hit the Continuum Analytics or Enthought sites for their full-stack Python installations, even texts which are not about numerics.
On your questions:
1) I don't care about threading. I care about pushing as many compute bits through the Xeon as I can in a given amount of seconds (using Numpy). But as I am a data scientist, I need the REPL. C is out. Why can I still not do this easily? Multiprocessing is there, sure, but it's been unch for years, while Cuda, OpenCL etc are far too hard for guy like me whose intellectual bandwidth is occupied with the domain, not the CS. Isn't that what Python was supposed to be about? Getting stuff done? Why isn't Python vectoring my data through the CPU and GPU yet, 15 years after numeric was first introduced?
2) Continuum Analytics is doing an awesome job and I don't see why they, or Enthought, couldn't take up the mantle, 10gen/Datastax style to use a database analogy. They really know their customers, and the Continuum stack delivers real new value every 6 months, and not only for a scientific audience. More generally, real users in real domains should be driving the project.
3) I am less concerned about Python 3 happening as planned, than I am about the focus of the project. Type annotations? This is an intellectual indulgence if it does not increase performance. Asyncio? We've had async libraries for years! Even when I started Python we had async libraries (not as good but they were there). How is async something fantastic and new? It's nothing but polishing an existing capability a little bit further. Unicode. fine. But again, web focused. Nobody else cares. Xrange laziness. Okay. Leaves me cold. Print(). No CS benefit, but huge marketing loss as you can't go out to newbies anymore and say "hey, check this out... Python hello world?"
>>> print "hello world"
"hello world"
So. What is Python. A serious language? NO. A wonderfully malleable, not too serious, friendly language, into which you can insert some real hardcore stuff (Numpy, ML, website parsing, database transformations, game scripting, image processing....the list is endless) really easily? Yes.
Where is Python genuinely way ahead of everyone else? Only on numerical computing.
It strikes me that Guido and co are embarrassed by their weekend hack of 1.x and 2.x, when that is precisely what the user base loves about it. Their attempt to make Python serious, is killing Python's original spirit. There is nothing wrong with 2.7. Nobody wants Python to morph into Java.
So, am I a Luddite wanting 2.7 to live forever? no. What I want is vectorization plus DAG-like workflows. These are the most important pieces of computer science that actually dovetail with real world use cases, today. Yes async is cool, but golang now owns that space. What I'd really like is for Python to give us a good framework for the Big Data world which is in almost everybody's use case now, and that means, Python needs to talk multiprocessor, Python needs to talk GPU, Python needs to talk cluster, and Python should long ago have been addressing this directly. Python needs to "go vectorized". That should be the project's obsession. SIMD, in a word, where the parallel granularity can go from GPU kernels,to CPU, to multi-machine clusters, and DAG-like workflows (with possible recursion) built in. This is not a nice-to-have capability anymore. It is what the next wildly popular mainstream programming language will have, built into the language. The signals from everything we're seeing added to 3.x is NOT this. It's web-like stuff. That battle is over. JS won.
Let's not gift the opportunity of huge data to Java (Spark) and Cuda, or a new language that will see the future better than us. It should be Python!
There you go. My view.
No, that's not where it is. It's in being a user friendly language for people who need to get numerical work done, and there's a huge difference. IPython/Jupyter did not happen in any other language, not even remotely (the closest thing I've seen is some Lua graphic repl that didn't go far).
> Nobody wants Python to morph into Java.
Python is not morphing into Java, far from it. 3.x represents a cleanup of the language (str vs. unicode, new vs. old style classes, a lot of other stuff). I do not agree with all of it (I too disagree with demoting print from statement to function), and there is other stuff I would have done - but most of it was desperately needed.
> What I want is vectorization plus DAG-like workflows. These are the most important pieces of computer science that actually dovetail with real world use cases, today
For a tiny, tiny part of the users. And they already have e.g. Theano, and a few other tools, that give some solutions.
> What I'd really like is for Python to give us a good framework for the Big Data world which is in almost everybody's use case now,
Excuse me, but I have to disagree anyone other than a tiny (perhaps 1 percent) of Python users care about big data. Most definitely not "everybody's use case". And the minority which actually cares about big data should not get anywhere close to Python - the runtime is prohibitive.
> This is not a nice-to-have capability anymore. It is what the next wildly popular mainstream programming language will have, built into the language.
Interesting. What language has this today? I know not of a single one, mainstream or niche. I don't believe that you are right on this.
> Let's not gift the opportunity of huge data to Java (Spark) and Cuda, or a new language that will see the future better than us. It should be Python!
Check out Nim + Nimborg. You'll get the best of Python and Nim, and Nimborg would let you do the transition smoothly.
Your idea of Python is not Guido's idea of Python, and as far as I know does not have popular support anywhere. If I were feeling so strongly about my favorite language being so misguided, I would look for a new language. The momentum of existing development direction is essentially unstoppable, whether it is right or wrong.