New GitHub Tool Lets Coders Build Software Like Bridges

Seems like native advertising

is there a generic name for this type of thing? not sure how to google for existing implementations

This [1] is a good writeup on how they used it to replace the merge that github uses. You can't really capture the totality of corner cases in unit tests, especially for complicated things.

[1]: http://githubengineering.com/move-fast/

"100% test coverage" doesn't imply "won't cause real-world things to break". That your tests run all the code doesn't mean that your tests cover all the actual cases. This identifies bugs and performance regressions on a wide assortment of real-world cases.

Agreed that writing test coverage and running twin systems against would get you this. But the benefit is that when you have no or low test coverage, if you have a drop in way of just testing behavior matching then it's probably a more economical approach in many projects. Also, I doubt any test suite exists with the variance of inputs that live user testing generates, including error cases.

I was wondering the same, but I guess the advantage is being able to safely-ish run tests in production against production data and other live services

the sort of stuff you usually mock out in unit tests. and may not be feasible to have a whole copy of your prod 'big data' and services just for testing on

with Scientist you get a report not just of mismatched results and exceptions (stuff good tests should usually be able to catch) but also if the new code is slower

the name is a bit annoying. seems very dude-cool-name-syndrome.

Yes, I was imagining a "traffic demultiplexer" of some sort that could feed the old and new backend in parallel and observe they have the same behaviour.

I think something like that would be useful for a lot of people in industry who have to maintain legacy monoliths. The ideal tool would allow "switchover" from old to new gradually, one URL endpoint at a time.

This can be used to replay traffic https://github.com/buger/gor/ so can do load testing at least...

Scientist seems to run both versions of the code.

Lot of comments here attacking the analogy. It's just a throwaway analogy. Can we just overlook non-essential parts of the article and discuss the code and techniques instead?

> On June 14, 1874, John Robinson led a "test elephant" on a stroll across the new Eads Bridge to prove it was safe.

https://en.wikipedia.org/wiki/Eads_Bridge

You have to conflicting issues on this point.

The first being that the code works fine, the deployment works fine, everything is working just fine. Do. Not. Touch. It.

These are some of the most financially important systems in America to date. It's not a question about if they have to work, it's a question about just how bad things would get if they didn't work for even a day.

The next problem that comes up is what happens when someone finds a bug 60, 40, or even 10 years in the future? What happens when some of this old hardware breaks and it becomes not only impossible to find replacement parts but also impossible to fabricate new ones in house.

Eventually, like all languages, Fortran will die out as time goes on. So few people know it now that it is inevitable.

Eventually something has to give and we can do two things until then.

1. Wait for time to pass and pretend nothing will ever happen to these machines.

2. Slowly start rewriting all of the stack, and transferring everything to more modern systems designed to be just as long term as their predecessors so that in the event of a replacement being needed it exists.

I don't know about anyone else, but I like to have backups/fallbacks when planning for the future.

I'm not saying that Ruby or any new language should be entrusted with this kind of responsibility. But there needs to be a decision and it needs to be made over the next couple of decades.

I used to work at a TBTF financial institution. During orientation, they told us that their mainframes process roughly 1/3 of US credit card transactions. The low-level code is 40-year old Fortran that still performs swimmingly.

Yeah, this scared me.

I'm in webdev, and while it does have its worrying trends with regards to the larger trade of programming, they are largely insular and contained. Only way to explain this kind of behavior is to have worked your entire career in webdev (with horse blinders on) and have next-to-no understanding of computer science. That, or she doesn't give a shit because Github is paying her a fuck-ton not to.

I'm going to guess in this case, it's the latter.

To be fair, developing something like this is probably orthogonal to developing the kinds features they have been neglecting to develop recently. Scientist was probably entirely within engineering, and not a massive effort. Making the things users are asking for would be a much bigger effort across the organization.

But, I do completely agree that they've been dropping the ball lately.

This is Github open-sourcing an internal tool, which is something they have a long history of doing (Jekyll, Resque, Hubot).

Most dev shops have a whole bunch of things like this -- assorted libraries and internal tools that are 80-90% of the way to being somewhat useful to the public. It's fantastic that Github provides the time/space/expectation for employees to finish them.

I don't see how this is relevant. Do you want to see Github implement your favorite feature, but at the same time the bugs introduced in the new version breaks the existing functionalities? That's what this library does. It ensures new bugs do not take down existing functionalities.

Multi-version execution is an active area of research [1].

[1]: http://srg.doc.ic.ac.uk/files/papers/mx-icse-13.pdf

I remember seeing this project more than a year ago (I believe it was call dat_science back then).

>> We can infer a shift from GitHub to alternate services soon from this, unless they change.

Nope. People are lazy. Will a lot of people write comments and all, how they are going to totally move away from github? Oh yeah, no doubt. But will they actually do it? Nah, just a few, unless github becomes as bad as source forge but I don't think it's going to happen, even then it would take some time.

Yes, yes, I am aware that migrating is just a matter of adding a new origin. That's the easy bit. But getting used to a new interface and everything, that takes much more effort and time. Writing a comment is easier (especially when tons of people are doing that around you) than changing your habits.

Even ignoring everything above, I don't really agree with the "puts effort into stuff like this as opposed to actually listening to their users" attitude. GitHub is not a one man's project. The whole company shouldn't drop everything just to do something right now because it was trending on HN a week or two ago. A company like this (making substantial product changes rather urgently based on some hype) maybe sounds cool but, in reality, you would probably not be excited to use their products. Urgent changes in features and quality often does not go together.

"The greatest form of flattery is imitation."

Huh, does that apply less if they've basically never heard of you when they imitate you? Not to imply that Erlang is unheard of, I'm just saying it's not as popular and orthodox as Python/Ruby/C/java/etc.