jbbarth

How is the POSIX non compliance a major problem? Does it have practical impacts when writing scripts that are not meant to be portable to other systems/shells?

Wow thanks for the completeness of the answer.

I found myself nodding on all the conceptual limits and features you added, I'm sold, gonna try Cadence asap :)

The tooling around SWF (web console and ability to get insights about tasks, failures, etc.) is definitely a big one from an operational perspective. The SWF console is indeed absolutely terrible (with basic bugs not fixed for years, like broken pagination), so we ended up developing our own here at Botify, along with a python based client lib that mimics most of RubyFlow principles. I'm curious if all this can be integrated with Cadence, will have a look. I can keep you informed if you feel it's valuable for the Cadence project.

Cadence looks like an OSS version of Amazon Simple Workflow (SWF) service. The author used to work on SWF at AWS afaik.

I'm a heavy SWF user at work for managing complex data pipelines. SWF requires an important conceptual and tooling effort in the beginning, but it gets reimbursed if you use it a lot.

As for other comments mentioning Airflow: the programming model is quite different , since Airflow as far as I understand forces to provide a DAG of tasks upfront. SWF (and Cadence?) doesn't, it coordinates the work of Deciders and Activity Workers and only acts as a source of truth for the state of the workflow (+ distribute task in a unique manner to many long-polling workers). As a result you don't declare anything upfront and can have deciders take dynamic decisions along the way, which is really nice when you want very dynamic logic for your workflows (e.g. dynamic partitioning of tasks, decisions depending on external factors, etc.).

I'd love to have Maxim insights about how Cadence compares to SWF, and what would be the reasons/challenges behind migrating from SWF to Cadence for SWF users (except that SWF is basically stale for 4+ years and rigged with arbitrary limits)

Maybe take a look at https://github.com/jordansissel/pleaserun

Don't throw the baby out with the bathwater maybe? Here's a fixed version of your statement:

"The default Turbolinks progress bar that can be removed with one line of CSS gives the impression that things are slow on slow connections."

Turbolinks is not perfect and has some drawbacks, but it also makes things faster in 99% (100%?) cases in my experience for almost 0 development effort.

Unfortunately this workflow is pretty fragile and it will not protect you from any of pip locking shortcomings.

It will basically work only in favorable cases where you have no ambiguous choice, no conflict (A depends on B==1.2.3, C depends on B==2.3.4), no reinstall to consider (git VS non git deps for instance), ... Really pip-tools is not that bad for now, until pip itself moves toward industry standards by default (lock file, deps resolution, reliable version checks, ..)

It seems there's an assumption in the article and in the comments here that "more EVs == net positive impact on pollution / greenhouse gases". This is not obvious at all. It depends on the way you produce electricity, and when 75% of it comes from coal, you may end up worsening the problem. This could be a smart move if the automobile market had a bigger inertia than the energy one, but I doubt it's the case...

Patches from upstream are regularly backported by distribution vendors, including Redhat, Debian, Ubuntu (though I don't know if Ubuntu do this themselves or base on Debian). Anyway, looking at the version number is not enough, you should refer your vendor security advisories.

Context: working at a company heavily relying on AWS. Made a request for such credits yesterday.

It reflects their SLA policies and docs, no surprise here. My AWS sales contact confirmed there's no problem with such demands (and basically they won't waste your time asking for failing X-Request-Id's or deny there was a problem).

What will be more interesting is how they'll deal with credit demands on all the S3-dependent services that were down at the same time. My company AWS bill is only 20% of S3@us-east-1, so we were completely down for 4+ hours and can only claim for 2% of our previous bill (woohoo).

But if we add Cloudfront, part of EC2, SWF, etc. it can become a different story.

It's likely that with the feature set growing, a version that uses a slightly more advanced language would be easier to maintain and have less "traps" than the bash version...

I'd be happy to see your 10 lines bash version of this ruby tool, with the same options/properties (debug messages, customizable prompt, history logging, same safeguards, rlwrap support, man page).

Not the answer you would expect but still: it can also happen on large projects where the refactoring or the paradigm change is just too costly in C. The pure, raw ability to make C faster is nice, but if it takes you months of development you don't have, it's pretty useless.

One typical example of this was a few years ago (if I'm not mistaken) in the monitoring world, when Shinken released a Nagios-compatible engine in Python, and, basically the reactions in the Nagios community was that the modifications involved in Nagios (C) were just too important to be worth it.

Didn't know that option thanks. Better than nothing, but unfortunately your environment is still subject to the remarks in my second paragraph. So pip-tools is still required if you want more guarantees.

I don't get why "pip" is checked in "Has separate manifest and lock files". Actually it doesn't have that feature (which Bundler for Ruby has, for instance). This very feature comes with a third-party package called "pip-tools", or possible alternatives, but raw pip doesn't have this ability directly afaik.

Fwiw pip isn't even able to enforce versions correctly (packages are installed as the file is read, and can conflict with previously expressed constraints). Or report installed versions correctly (it's possible that packages are half-installed or installed but not reported as such by pip commands).

The article doesn't mention "45 sales people" for Dishero, but for the previous company of their "VP of sales". Curious about the number for Dishero tho.

This argument often comes up and I'm genuinely curious why people think Django is less "magic" than Rails, in my experience it has been the opposite.

My experience with Django (for instance: class-based views and admin parts) is actually that you have dozens of magical classes or methods and if you don't know them all, you're screwed. Often when I try to do something, many aswers on StackOverflow boil down to "just override a_method_that_is_hidden_somewhere()" or "just use ListButWithSpecificBehaviorView(), easy you see?".

Are there any specific example you have in mind where Rails do something magical that requires more code in Django but where Django is more explicit?

PS: not trying to start a flamewar, both are OK-frameworks, bla bla bla ;-)

bundler has many more features, for instance it provides a sandboxing mechanism and it allows for breaking the app boot if a dependency is not correct (which is a super good thing, I can't count the number of times I had a different egg than another developer and it was breaking things...)

edit: you're totally right tho! that was just a complement

Have a look at pip-tools https://github.com/nvie/pip-tools which does basically that and a bit more.

The point is that everything at scale as an impact that looks big when aggregated... if you don't actually look at the scale.

135 out of 1B isn't big at all, it just looks big because of the biases we have when interpreting big numbers. Not mentioning the fact that the aggregation isn't very relevant (it's not like 135 people will have their entire life wasted while the others are not annoyed at all).

This calculus doesn't make much sense. Multiply any enormous number of people by a few minutes and you'll get hundreds of lives equivalent time.

That's pretty much the kind of fallacy behind "if all people on Earth give 10$ for <cause> we can solve <big problem mankind hadn't solve in a century>."

Actually on some parts of the world it's not 12:00AM you know, and there are sysadmins/devops there too :-)

I'd say Python tooling may be OK at language level, even slightly better (code introspection abilities are more recent in ruby I think, the fact you could get method docs, method def, etc.). The equivalent for IPython (shell version, not notebooks) in Ruby is pry and it's pretty good.

On the other side, in Python higher-level tools are horrible from an operational perspective, and much worse than Ruby's if you ask me (pip vs bundler, rbtrace vs.. gdb-python(?), pry/pry_remote vs pdb, ...).

Also +1 on boardwaalk's comment, this side of Python is pretty good.

The article makes some very good points, and I'm surprised you don't talk more about the deploy advantages for instance. But I was a bit annoyed by little things that I think are inexact:

- the "one-process-per-request" meme along the post applies only to some ruby app servers (there are event loop and threaded models too, think thin, puma, passenger in some modes) and I guess reading between the lines that it's mostly a problem of thread-safety and async support, because of the gems Parse used to have, right? I'm sure that limits options at some point anyway, but the statement is misleading and not really explained, I'd love to hear more details

- I don't understand how the comments in the little Go file snippet applies in any way to "ruby" ; it may be rails caching mechanisms, or a specific gem, but I have a hard time mapping those very specific details to something intristic to ruby, it seems more like grumpy ruby bashing, like you'd have done php bashing 5 years ago

As all rewrite stories, I think there's a part of envy/excitement over the new cool tech you want to use (and that's fair! pleasure give you huge productivity boosts), and also a part of success related to the fact you know the kind of things you failed in the first version, so you won't make the same mistakes the 2nd time.

I'd love to hear finer details on those points! Great article overall anyway

In Ruby some people achieve this by wrapping actions in their own class, which does the trick if the classes are kept independent from each other (no shared state, except global vars, which nobody uses in the ruby world).

Is it? Having it hosted on my servers doesn't make it decentralized: it's just centralized in a place I own. If the server(s) go down, same problem as Github (except I may be able to take action). Right?

Would you share how you did that technically in Python? Maybe some code example or a lib to recommend? decorators? assertions?