crimsonalucard1

Doesn't the net CO2 in the air increase?

So you harvest CO2 to into fuel. That fuel is burned and the CO2 is released back into the air. So net difference is zero.

But to harvest the CO2 you needed to generate a high amount of heat. That's extra CO2. So net CO2 is more.

It depends on the cost of harvesting that CO2.

>If you say IO is the bottleneck, then you're claiming there is no significant difference between python and node. That's what a bottleneck means.

This is my claim that this SHOULD be what's happening under the obvious logic that tasks handled in parallel to IO should be faster then tasks handled sequentially and under the assumption that IO takes up way more time then local processing.

Like I said the fact that this is NOT happening within the python ecosystem and assuming the axioms above are true, then this indicates a flaw that is python specific.

>The reason the interpreter matters is because the latency picture of async is very linear:

I would say it shouldn't matter if done properly because the local latency picture should be a fraction of the time when compared to round trip travel time and database processing.

>Python is likely slow because it implements the event loop in python

Yeah, we're in agreement. I said it was a python specific problem.

If you take a single task in this benchmark for python. And the interpreter spends more time processing the task locally then the total Round trip travel time and database processing time... Then this means the database is faster than python. If database calls are faster then python then this is a python specific issue.

I'm hoping that other then treating diseases this thing can aid with enhancement.

I think you need to pause before considering this route. Are you that exceptional? This level of intelligence is incredibly rare and unless you're 100% sure you're capable of this, I wouldn't do it.

>Unless you've actually profiled the thing and shown where the time is used, all your assertions are nonsense.

It's data science that is causing this data driven attitude to invade peoples minds. Do you not realize that logic and assumptions take a big role in drawing conclusions WITHOUT data? In fact if you're a developer you know about a way to DERIVE performance WITHOUT a single data point or benchmark or profile. You know about this method, you just haven't been able to see the connections and your model about how this world works (data driven conclusions only) is highly flawed.

I can look at two algorithms and I can derive with logic alone which one is O(N) and which one is O(N^2). There is ZERO need to run a benchmark. The entire theory of complexity is a mathematical theory used to assist us at arriving AT PERFORMANCE conclusions WITHOUT EVIDENCE/BENCHMARKS.

Another thing you have to realize is the importance of assumptions. Things like 1 + 1 = 2 will remain true always and that a profile or benchmark ran on a specific task is an accurate observation of THAT task. These are both reasonable assumptions to make about the universe. They are also the same assumptions YOU are making everytime you ask for EVIDENCE and benchmarks.

What you aren't seeing is this: The assumptions I AM making ARE EXACTLY THE SAME: reasonable.

>you're asserting something without actual evidence, and the article itself doesn't actually state that either: it contains no breakdown of where the time is spent

Let's take it from the top shall we.

I am making the assumption that tasks done in parallel ARE Faster than tasks done sequentially.

The author specifically stated he made a server that where each request fetches a row from the database. And he is saying that his benchmark consisted of thousands of concurrent requests.

I am also making the assumption that for thousands of requests and thousands of database requests MOST of the time is spent on IO. It's similar to deriving O(N) from a for loop. I observe the type of test the author is running and I make a logical conclusion on WHAT SHOULD be happening. Now you may ask why is IO specifically taking up most of the time of a single request a reasonable assumption? Because all of web development is predicated on this assumption. It's the entire reason why we use inefficient languages like python, node or Java to run our web apps instead of C++, because the database is the bottleneck. It doesn't matter if you use python or ruby or C++, the server will always be waiting on the db. It's also a reasonable assumption given my experience working with python and node and databases. Databases are the bottleneck.

Given this highly reasonable assumption, and in the same vein as using complexity theory to derive performance speed, it is highly reasonable for me to say that the problem IS PYTHON SPECIFIC. No evidence NEEDED. 1 + 1 = 2. I don't need to put that into my calculator 100 times to get 100 data points for some type of data driven conclusion. It's assumed and it's a highly reasonable assumption. So reasonable that only an idiot would try to verify 1 + 1 = 2 using statistics and experiments.

Look you want data and no assumptions? First you need to get rid of the assumption that a profiler and benchmark is accurate and truthful. Profile the profiler itself. But then your making another assumption: The profiler that profiled the profiler is accurate. So you need to get me data on that as well. You see where this is going?

There is ZERO way to make any conclusion about anything without making an assumption. And Even with an assumption, the scientific method HAS NO way of proving anything to be true. Science functions on the assumption that probability theory is an accurate description of events that happen in the real world AND even under this assumption there is no way to sample all possible EVENTS for a given experiment so we can only verify causality and correlations to a certain degree.

The truth is blurry and humans navigate through the world using assumptions, logic and data. To intelligently navigate the world you need to know when to make assumptions and when to use logic and when data driven tests are most appropriate. Don't be an idiot and think that everything on the face of the earth needs to be verified with statistics, data and A/B tests. That type of thinking is pure garbage and it is the same misguided logic that is driving your argument with me.

>Well, it's odd that you say that, when previously you were claiming that it was caused by Python. Is it caused by Python, or is Python negligible?

It's not odd. Think harder. I'm saying under the benchmark and according to the logic of what SHOULD be going on under AsyncIO it SHOULD be negligible. So such performance issues between python and node SHOULDN'T matter, and that's why you CAN compare NodeJS and Python.

But actual testing does show an unexpected discrepancy that says that the problem is python specific because logically there's No other explanation.

>You're really sure that there's no context where the accuracy gained by relativity would be useful?

Dude I already know about that. I just didn't bring it up, because I'm giving you an example to help you understand what "NEGLIGIBLE" means. You're just being a pedantic smart ass.

There are many many cases where relativity is not needed because it's negligible. In fact the overwhelming majority of engineering problems don't need to touch relativity. You are aware of this I am aware of this. No need to be a pedantic smart ass. The other thing that gets me is that it's not even anything new, many people know about how satellites travel fast enough for relativity to matter.

>Again, this is you making stuff up

Dude nothing was made up.

I never said "there's no context where the accuracy gained by relativity would be useful". "No context" is something YOU made up.

In fact "made up" is too weak of a word. A better word is an utter lie.

That's right. You're a liar. I'm not being rude. Just making an observation. Embarrassed yet?

> I just think that your comparison with _node.js_ when there are a bunch of confounding variables is nonsense

And I'm saying all those confounding variables you're talking about are negligible and irrelevant.

Why? Because the benchmark test in the article is a test where every single task is 99% bound by IO.

What each task does is make a database call AND NOTHING ELSE. Therefore you can safely say that for either python or Node request less than 1% of a single task will be spent on processing while 99% of the task is spent on IO.

You're talking about scales on the order of 0.01% vs. 0.0001%. Sure maybe node is 100x faster, but it's STILL NEGLIGIBLE compared to IO.

It it _NOT_ Nonsense.

You Do not need an apples to apples comparison to come to the conclusion that the problem is Specific to the python implementation. There ARE NO confounding variables.

>Your logic makes perfect sense, in a world where I/O bound processes, JIT versus interpretation differences, garbage collection versus reference counting differences, etc., don't exist. But those things do exist in the real world, so if your logic doesn't include them, you're quite likely to be wrong. In general, an interpreted concurrent system is far too complex to make performance predictions about based only on logic, because your logic can't possibly include all the relevant variables.

Hey Genius. Look at the test that the benchmark ran. The benchmark is IO bound. This is SPECIFIC test about IO bound processes. The benchmark is not referring to real world applications it is SPECIFICALLY referring to IO.

The literal test is thousands of requests and for each handler for those requests ALL it does is query a database. If you look at a single request almost 99% of time is spent on IO.

Due to the above, Anything that has to do with the python interpreter, JIT, garbage collection and reference counting becomes NEGLIGIBLE in the context of the TEST in the ARTICLE ABOVE. I suspect you didn't even read it completely.

Does that concept make sense to you? You can use relativity rather then newtonian physics to calculate the trajectory of a projectile BUT it is involves UNNECESSARY overhead coming from Relativity because the accuracy gained from the extra calculations ARE NEGLIGIBLE.

>You are, in fact, making stuff up. If you are offended by accurate description of your behavior, behave better.

Now that things have been explained completely clearly to you, do you now see how you are the one who is completely wrong or are you incapable of ever admitting your wrong and apologizing to me like you should? I mean literally that statement above is embarrassing once you get how wrong you are.

Except the problem here is that those tests were bottlenecked by IO. Whether you're testing C++, pypy, libuv, or whatever it doesn't matter.

All that matters is the concurrency model because that application he's running is barely doing anything else except IO and anything outside of IO becomes negligible because after enough requests, those sync worker processes will all be spending the majority of their time blocked by an IO request.

The basic essence of the original claim is that sync is not necessarily better than async for all cases of high IO tasks. I bring up node as a counter example because that async model IS Faster for THIS same case. And bringing up node is 100% relevant because IO is the bottleneck, so it doesn't really matter how much faster node is executing as IO should be taking most of the time.

Clearly and logically the async concurrency model is better for these types of tasks so IF tests indicate otherwise for PYTHON then there's something up with python specifically.

You're right, we are in disagreement. I didn't realize you completely failed to understand what's going on and felt the need to do an apples to apples comparison when such a comparison is not Needed at all.

As a general rule it works. Work friends are not friends. There are exceptions but don't expect anything.

Except IO is the bottleneck here. The concurrency model for IO should determine overall speed. If python async is slower for IO tasks then sync then that IS an unexpected result and an indication of a python specific problem.

When I talk about async await I'm talking about everything that encompasses supporting that syntax. This includes the I/O event loop.

So really we're in agreement. You're talking about reimplementing python specific things to make it more performant, and that is exactly another way of saying that the problem is python specific.

Bottleneck is IO. Concurrency model should be the limiting factor here.

NodeJS is faster than flask because of the concurrency model and NOT because of the JIT.

The python async implementation being slower than the python sync implementation means one thing: Something is up with python.

The poster implies that with the concurrency model the outcome of these tests are expected.

The reality is, these results are NOT expected. Something is going on specifically with the python implementation.

NodeJS primitives are enough to produce the same functionality as flask without the need for an extra framework.

Right but this test focused on concurrent IO. The bottleneck is not the interpreter but the concurrency model. It doesn't matter if you coded it in C++, the JIT shouldn't even be a factor here because the bottleneck is IO and therefore ONLY the concurrency model should be a factor here. You should only see differences in speed based off of which model is used. All else is negligible.

So you have two implementations of async that are both bottlenecked by IO. One is implemented in node. The other in python.

The node implementation behaves as expected in accordance to theory meaning that for thousands of IO bound tasks it performs faster then a fixed number of sync worker threads (say 5 threads).

This makes sense right? Given thousands of IO bound tasks, eventually all 5 threads must be doing IO and therefore blocked on every task, while the single threaded async model is always context switching whenever it encounters an IO task so it is never blocked and it is always doing something...

Meanwhile the python async implementation doesn't perform in accordance to theory. 5 async workers is slower then 5 sync workers on IO bound tasks. 5 sync workers should eventually be entirely blocked by IO and the 5 async workers should never be blocked ever... Why is the python implementation slower? The answer is obvious:

It's python specific. It's python that is the problem.

The database is the bottleneck. JIT or even C++ shouldn't even be a factor here. Something is wrong with the python implimentation of async await.

Yep and this test being shown is actually saying that about 5 sync workers acting on thousands of requests is faster then python async workers.

Theoretically it makes no sense. A Task manager executing tasks in parallel to IO instead of blocking on IO should be faster... So the problem must be in the implementation.

>And no I'm not interested in whatever benchmark you're going to want to post,

That's rude.

Let's put it this way. NodeJS and nginx leveled the playing field. It destroyed the lamp stack and made async the standard way of handling high loads of IO. From that alone it should indicate to you that there is something very wrong with how you're thinking about things.

You know the theory of asyncio? Let me restate it for you: If coroutines are basically the SAME thing as routines but with the extra ability to allow tasks to be done in parallel with IO then what does that mean?

It means that 5 async workers in theory should be more performant than 5 sync workers FOR highly concurrent IO tasks.

The logic is inescapable.

So what does it mean, if you run tests and see that 5 async workers are NOT more performant than 5 sync workers ON PYTHON exclusively? The theory of asyncio makes perfect logical sense right? So what is logically the problem here?

The problem IS PYTHON. That's a theorem derived logically. No need for evidence or data driven techniques.

There's this idea that data drives the world and you need evidence to back everything up. How many data points do you need to prove 1 + 1 = 2? Put that in your calculator 200 times and you got 200 data points. Boom data driven buzzword. That's what you're asking from me btw. A benchmark, a datapoint to prove what is already logical. Then you hilariously decided to dismiss it before i even presented it.

Look, I say what I say not from evidence, but from logic. I can derive certain issues about the system from logic. You just follow the logic I gave you above and tell me where it went wrong and why do I need some dumb data point to prove 1+1=2 to you?

There is NOTHING made up above. It is pure logic derived from the assumption of what AsyncIO is doing.

>But note how I said "probably" because I don't know for sure, and I'm not comfortable with making things up and stating them as facts.

But you seem perfectly comfortable in being rude and accusing me of making stuff up. I'm not comfortable in going around the internet and trashing other peoples theories with accusations that they are making shit up. If you disagree say it, I respect that. I don't respect the part where you're saying I'm making stuff up.

No man. Nodejs will beat the flask benchmark. For this specific test there is no downside to async.

What’s going on here is python specific.

This is not what is happening with flask/uwsgi. There is a fixed number of threads and processes with flask. The threads are only parallel for io and the processes are parallel always.

Yeah except nodejs will beat flask in this same exact benchmark. Explain that.

I see this elitist attitude all over the internet. First it was people saying “Guys why are you over reacting to corona the flu is worse.”

Then it was people saying “Guys, stop buying surgical masks, The science says they don’t work it’s like putting a rag over your mouth.”

All of these so called expert know it alls were wrong and now we have another expert on asynchronous python telling us he knows better and he’s not surprised. No dude your just another guy on the internet pretending he’s a know it all.

If you are any good, you’ll realize that nodejs will beat the flask implementation any day of the week and the nodejs model is exactly identical to the python async model. Nodejs blew everything out of the water, and it showed that asynchronous single threaded code was better for exactly the test this benchmark is running.

It’s not obvious at all. Why is the node framework faster then python async? Why can’t python async beat python sync when node can do it easily? What is the specific flaw within python itself that is causing this? Don’t answer that question because you don’t actually know man. Just do what you always do and wait for a well intentioned humble person to run a benchmark then comment on it with your elitist know it all attitude claiming your not surprised.

Is there a word for these types of people? They are all over the internet. If we invent a label maybe they’ll start becoming self aware and start acting more down to earth.

Dynamic code upgrades? Can you explain what that is and why a type system will prevent that from happening?

Yeah but usually that's not the case. Usually you can get it back.

I would say more often than not it's reversible.

It is reversible. If you make a mistake like this oftentimes you can rely on law enforcement/bank to force the money to be given back. Not sure if this is possible with crypto.

I'm waiting for the day where 3D printers become so powerful that I only need a minimal set of these "physical" DIY skills to build random physical objects.

Anybody know of a printer that can get me very close to this goal?

>but up close the details of say, managing state with hooks in React, is very different from managing state with services and RxJS observables in Angular.

I would say learning the detailed difference when needed for the project is in fact extremely trivial. The wrong kind of learning is thinking that you're improving your skills by memorizing and learning all of these details even when it is not required by your current project.

I would say max it takes about 2 weeks to a month for a developer to get good with react. It's not even a big deal if you think about it, but hiring managers still look for "relevant experience."

Don't learn the details unless it's relevant to the project. To improve your overall skills as a developer: Learn the fundamentals.

Though I have noticed one thing. Many recruiters and even technical hiring managers are buzzword driven. Buzzwords are often related to details so to pad your resume you have to learn the details. Ultimately though, if you end up never touching that detail or that detail changes then you wasted your time learning it.

Learn the fundamentals because the fundamentals never change.

Honestly you're the kind of developer that the OP is talking about, but then again most developers are like that.

>But if a candidate is motivated to take a job because they get to use shiny new technology, i would suggest that they are not, in fact, a good candidate.

I would argue that this attitude often exists at the corporate level as well.

Most companies I've worked exist in a perpetual state of migration from monolith to Micro services. Basically the overall architecture remains static in the sense that it's always a giant monolith sitting in the center with some satellite services surrounding it.

The migration to microservices and kubernetes is 99% buzzword driven and rarely has nothing to do with whether or not the product will benefit from the migration. The weirdest part about it is that it's many times a never ending thing initiative. It's very rare for a company to actually arrive to a point where every component of their system is split out into small pieces. It usually just stays permanently as some services surrounding a monolith despite all the talk.

When entire companies are like this it's usually a sign that the entire company is weak. My experience has been that many companies have this attitude and many programmers share it.

If all/most programmers share this general attitude then I wouldn't call these "new technology chasers" bad as most people are like this.

>In my experience, magpieism is associated with weaker programmers; not the very worst, but second quartile, say - people who get very engaged with superficial details, but don't think about deeper things.

I may be biased because I know I talk a lot. But my experience has actually shown that this is not true.

People who don't know a lot of things tend to stay silent. This strategy works consistently to allow many people to hide extraordinary gaps in knowledge. It's very common and the strategy works.

When you see a person not talking too much... human bias tends to err on the fact that he's just not a talker or he may be deep in thought. Never do you think that a person isn't talking because he doesn't know anything. To be not biased is to consider all the possibilities and basically 90% of people just skip over the thought that the person isn't talking because he doesn't know anything.

Now for a talker, the bias goes in the other direction because a talker has an incredibly higher chance of saying something you disagree with or is flat out wrong. This chance is higher EVEN when the talker is more knowledgeable than the non-talker simply because the talker places himself on the pedestal of judgement every time he opens his mouth while the non-talker avoids it.

If you know a non-talker who may or may not have kept his trap shut after you explained something to him, really the only way to confirm whether he knows what you're talking about is to ask him confirmation questions after you talked to him. If you don't do this, likely he'll just google the info later and pretend that he always was on top of everything.

You have to really watch out for this bias. Even as a talker I tended to bias towards thinking that the quiet nice guy was more intelligent. The reality is, "magpieism" has no correlation to how good or bad someone is... if someone talks a lot they could go either way, it's just that because they talk a lot, you have a ton of info on how good they are based off of what they say... while the non-talker you have no info and your judgement is (unknowingly to you) influenced by your biased optimistic assumptions.

If someone is quiet there is a higher chance that he's quiet because he's totally lost. And if he's quiet often... then there's a higher chance that he's totally lost all the time.

This stuff all comes from China originally. I'm Chinese so I know.

It's because Chinese characters use to be written with brushes and with brushes the stroke order of the lines is very important for legibility and beauty. See the pictures in the link, you'll understand: https://www.google.com/search?q=chinese+calligraphy&rlz=1C5C...

Just imagine what those characters will look like if you did strokes out of order...

This stroke order carried over to pen and pencil and also to japan after they adopted Chinese characters as Kanji and invented their own unique scripts called hiragana and katakana.

Even with a pencil, stroke order is still very important to Chinese characters, it's kind of like an accent in your writing. People can sense something is off with your writing if you wrote strokes out of order. It's still legible and can even be neat but people can sense something off with stroke order.

Japanese people are just taking stroke order and applying it to English.