NPM and Left-Pad: Have We Forgotten How to Program?
haneycodes.net
haneycodes.net
var passAll = require('101/pass-all')
var isPositive = require('is-positive')
var isInteger = require('is-integer')
module.exports = passAll(isPositive, isInteger)
They've already taken it to an entirely different level of insanity.
Really
Not that it was impossible, but I still remember having to search for it and being astonished that that was necessary.
Sure, should not apply anymore like that.
That's half the problem with these stupid micromodules; they're not abstracting complexity, they're obfuscating simplicity.
So many people on this page have written about how these are well tested, performant, and correct modules. But, the modules aren't even correct in many cases, let alone providing incomplete coverage over edge cases or the slow performance and horrendous dependencies of many of the modules.
Lodash does this (and versioning, and clean code, and tests) really well though.
I don't use NPM so maybe it doesn't really matter aside from the level of abstraction being implemented being relatively ridiculous.
However, if my build system had to go out and grab build files for every X number of basic functions I need to use, grab Y number of dependencies for those functions, run X * Y number of tests for all those dependent packages, AND then also fell apart if someone threw a tantrum and removed any one of those packages basically shutting me down for a day... then I'd question every single thing about my decisions to use that technology.
[Quick Edit] Basically I'm saying "Get off my lawn ya kids!"
Except, again, it's no more ridiculous than pick a random bit of code from anywhere and isn't any more emblematic than that random bit because, again, it's not used anywhere.
> [Quick Edit] Basically I'm saying "Get off my lawn ya kids!"
tl;dr: going for buzzfeed headlines but for comments, got it. Good job, I guess?
Those are just the three top-level dependencies. The package has 9 recursive dependencies.
There's also a nice table explaining how a package called "is-positive" managed to reach version 3.1.0.
https://www.npmjs.com/package/average
var average = require('average');
var result = average([2, 5, 0, 1, 25, 7, 3, 0, 0, 10]);
console.log('The average for all the values is:', result);
JavaScript has suffered from a lack of a standard library for a while. Having a small package like this means that (in theory) everyone is using the same bug free version of summing instead of writing their own.
Honestly having JS start building a standard library at this point would be wonderful.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
If you need something more convenient I highly recommend just adding lodash and getting a whole bag of rock-solid awesome tools.
And they won't let you have fixed-point in JavaScript!
Oh, if all your numbers are positive you can also sort the array and it will minimize the error accumulated, but I'm not sure what to do with mixed signs.
For example: https://www.npmjs.com/package/lodash.mean
Or, in light of `left-pad`: https://www.npmjs.com/package/lodash.padstart
You get the benefit of a "small, focused" module but still rely on a large and stable project with a great maintainer that cares about performance and testability.
https://github.com/bytespider/average/blob/master/src/averag...
and here is the code copied out in full
module.exports = function average(values) { for (var i = 0, sum = 0; i < values.length; ++i) { sum += values[i]; } return sum / values.length; };
is this worth adding a dependency to your build for?
Pretty much every package management system gets cruft in it like this. Example: for a long time someone had uploaded a random Wordpress core install into Bower.
The tradeoff is speed, especially for dynamically typed languages. Having to check your operand types in JS before doing work is a performance killer.
Don't fall into the 'C is the devil' trap, any tool can be misused.
There might be a few legitimate use cases for C, but I've seen people pick it for the wrong reason so often (and using C because "it would be performant" is entirely invalid IME).
Of course, static analysis is always used in combination with proper coding style... but that is just the normal (professional) C development environment.
> Of course, static analysis is always used in combination with proper coding style... but that is just the normal (professional) C development environment.
>> Straight-up C is not at all suitable for safety-critical software. C plus various bolt-on tools for static analysis and the like can be usable, but is always going to be less effective (IMO) than a unified language where every tool is working according to the same rules.
Pretty sure you've just restated GP's point in your second paragraph.
import fractions
print str(fractions.Fraction.from_float(0.3333).limit_denominator(10))
[0] http://www.thejach.com/pseudo-public/webgl_coursera/assignme...
[1] - https://en.wikipedia.org/wiki/Kahan_summation_algorithm
The problem with a naive average like this is that if you're averaging a bunch of big numbers there is a very high chance of overflow when you're summing, so even if all the numbers fit in 32 or 53 bits your calculation will be off.
If you're not averaging thousands of large numbers, why are you even using a package instead of nums.reduce((a,b)=>a+b) / nums.length anyway?
Just out of interest, what kinds of functions would you expect to have to write yourself, if you're not happy about calculating the average of a list of numbers?
If you had to, say, add a list of numbers together, without averaging them, would your first thought be to go searching for a package, given that you know some languages have a 'sum' function? Some languages have an 'add' function (separate from the operator) - would you go looking for a package to supply an add function if you (say) needed to pass it to reduce?
No, because if this isn't in the standard library (or a very simple one-liner from standard library functions like "fold +") then I don't want to be working in this language.
If I have to work in this language, and I'm allowed to bring in packages, I'd go look for the community's attempt at fixing the standard library, or at least a "math" package, particularly if there were lots of other basic math functions I'd have to do myself. If it's really just this, I'll probably paste a Stackoverflow answer.
Could I come up with it myself? Yes, but thinking through the pitfalls of the implementations of basic math operations is not a good use of time.
What would you do if your language didn't have exponentiation?
That you think averaging a list of numbers qualifies as a 'feature' you might add to a codebase is rather the surprising thing about your response. I'm aware I'm coming over rather arsey, and that isn't intended, I'm just surprised to find someone actually defend the attitude, generally, that says a trivial calculation should be either a) provided, or you're not going to use the language, or b) put in a package to mean you don't have to spend 'the time' writing it, while at the same time saying you don't care what is in those implementations.
I mean, it doesn't matter to me, obviously. You can choose to have whatever criteria for taking a job you like, and I can have whatever criteria I like for my hires. I'm just surprised. Sorry If I've come over argumentative or grandstanding.
Sorry, can I get this straight, we're talking about sum() now, right? I'm genuinely amazed.
> thinking through the pitfalls of the implementations of basic math operations is not a good use of time
I suppose there's a trade off, at some point it is less time to find, check, and invest in a package rather than write its contents. For lots of cases it is definitely faster to use a package. But we're talking about averaging a list of numbers here, or adding them, right? Doesn't this strike you as rather bizarre to be having this discussion over things you should be able to write trivially?
> What would you do if your language didn't have exponentiation?
It depends what I need it for and what kind of exponentiation. I'd use inline exponentiation by small positive integers. I'd be concerned if my team were writing pow(x, 2) + pow(y, 2), for example. If I needed fractional exponentiation (e.g. pow(k, 3.567)), then I know that's beyond the realm of something that could be implemented in a couple of lines. If the language didn't have it, I might write it, certainly, especially if it wasn't part of a bigger suite of functionality I need.
Sure, if it's really just arithmetic on lists, I won't be happy about it, but I'll write it. I've yet to meet a standard library which is wonderfully complete except for that one thing. If the abstractions of the environment I'm working in are that poor, there's going to be a thousand other little "trivial" things that I'm now responsible for maintaining, and some of them will turn out to be less trivial than imagined.
For example, it is "trivial" to write linked lists in C, but tracking down the one-character typo that causes them to nondeterministically explode is IMO a distraction from the project, not a valid part of it.
And what about the next project in that language? Wouldn't it be nice to factor out all that stuff and bring it with me? Well, now I am using a community standard library, but it's the community of "just me." Why not take one with some internet recommendations and an active Github?
My employer fortunately runs its own package mirrors, though, so we don't run the risk of packages just disappearing on someone else's whim.
I suppose our difference in values is that I consider each line of in-house code as much of a liability as you consider each line of other people's code.
My reaction to the article is very simple:
>What concerns me here is that so many packages took on a dependency for a simple left padding string function, rather than taking 2 minutes to write such a basic function themselves.
"So many people" writing a basic function themselves is an exactly wrong outcome IMO.
for n in list
sum += n
return sum / len(list)
l = len(list)
for n in list
sum += n / l
return sum
Then they correct it to
l = len(list)
for n in list
sum += n / l
rem += n % l
sum += rem / l
rem = rem % l
return sum
The best algorithm might be Knuth's which is fairly non-trivial and might not even be the fastest/most efficient given certain unique JS optimizations.
Do you want to do this for every 'trivial' function or would you rather import average where (in theory) someone knowledgeable has already done this work and benchmarks for you and you get the best performing algorithm basically for free?
2. The pursuit of all edge-cases is a form of hubris. Your first version is fine in many cases (it is the algorithm used by the package referenced [edit:] and by both the Python and .NET standard libraries) and can be written inline.
3. At the point you are concerned with specific edge-cases, and need the 'correct' algorithm, you need to have found that algorithm, understood it, and make sure that the package you're importing implements it. It's even worse for a builtin function: there's no guarantee that a language implementation of average does so, or that it has the correct tradeoffs for your code. You'd need to test it, code review it, and so on. Do you do that for every trivial function?
4. If you're really suggesting that you can't be trusted to calculate an average correctly, then how can you have the slightest confidence in anything you do that is more complex?
If you can't trust built-in functions to do what they say on the standard library documentation, you're either paranoid or made a wrong choice of language. (Or you have some very compelling reason to be working in a language that's still rough, like the early days of Swift, which is an entirely different game).
>If you're really suggesting that you can't be trusted to calculate an average correctly
It's not about what you can be trusted to do, but what's worth spending your time on.
Not going to check many, but a quick look, since I've got them handy: Python fails Spivak's criteria, as does .NET.
You examples does not mention the type of variables involved. And dependent on the type and the language your first example may be (a lot more) correct compared to your third example. Ie. negative floating point numbers.
But who says that a library implementation is a lot better than what you as a programmer can come up with?
In particular with all these "micro" libraries written by random people.
Are they actually smarter than you? Do they understand the "edge" cases as you do?
Most people don't bother with looking at the library source code they are importing. And I don't blame them because things are in general a mess and not only for JavaScript.
The solution, for me at least, is to import less and be more conversative with what you import.
It's going to be damn hard to make an average function sound impressive.
I can easily see this grow into a function accepting options indicating such things as "null values are zeroes", "ignore null values", "evaluate string arguments", etc, or maybe into a factory that produces customized averaging functions.
For example, average([null,1,2,3],[4,5]) could either be 3 (the average of all numbers), 2.5 (sum of numbers divided by number of 'root' items), undefined (input format error), 3.25 (average of average of [1,2,3] and average of [4,5]), etc.
And what if it gets passed arrays of equal size? "average([1,2,3],[7,4,5])" could easily be interpreted as a vector average, and produce [4,3,4]
Silly? Yes, but if you want to write a robust library in JavaScript, you have to think about what to do with all these kinds of input.
And of course, there are the simple variants of harmonic mean, arithmetic mean, and geometric mean, and several others (https://en.m.wikipedia.org/wiki/Average)
If an interviewee did respond with some or all of the points you raise he'd have turned a potential negative into a positive point. This instance is the simple arithmetic mean with no consideration of anything.
I stand by my original comment. Nine times out of ten an average is going to be resolutely unexceptional, one time in ten or less it's the core of a well-crafted solution with a decent reason for being.
Someone who understands what NPM is, has written and published working code (even stupid code) is miles ahead of the curve already.
Most companies in the world, especially non-software companies, don't go for 5% hacker gurus. Attracting someone vaguely competent is already a blessing.
Interviewed enough "C++, 3 years" people who couldn't even describe a class, or give the most basic description of OO, to have no illusions of some of the standards out there. Similar for other languages. Similar for how much having a degree is worth. I used to be surprised such people tried it on, as there was no way they'd walk out of the hour looking anything but stupid, but it happened often enough to stop surprising. Of course I've also experienced code produced by such types, and worked with a couple.
If you're going to publish something then make the world a better place not pollute it with pointless stuff.
Spending more time publishing this package than most basically competent developers would spend writing the four lines is not going to mark you as "miles ahead of the curve" in my eyes. If you're an 18yo fresh out of school without degree, seeking trainee role, I'd look on it favourably.
I wonder if there will be a 3rd release with an updated averaging function.
Edit: to be fair, it was an optimization from reduce to for loop [https://github.com/bytespider/average/commit/7d1e2baa8b8304d...]
ducks
(Edit: typo)
$ find node_modules/ -type f | wc -l
56278
$ find node_modules/ -type f | xargs md5sum | gawk '{print $1}' | sort | uniq | wc -l
8545
https://github.com/tjmehta/101/blob/master/pass-all.js
It’s written in such a way that every time you call...
passAll(f1, f2, ..., fn)(args..)
[From my quick eyeball count. These numbers could be a bit off.]
Instead, it’s a slow tower of bloat, which you need to read 5 files to reason about.
Javascript implementation:
function passAll() {
var fns = [].slice.call(arguments);
return function() {
for (var i = 0, len = fns.length; i < len; i++) {
if (!fns[i].apply(null, arguments)) { return false; }
}
return true;
};
}
passAll = (fns...) ->
for fn in fns then if typeof fn isnt 'function'
throw new TypeError 'all funcs should be functions'
(args...) ->
(return false if not fn args...) for fn in fns
trueI will probably have go back and change this now that I know about it. In general though, not gonna a lie, I am not very concerned about micro performance optimizations.
Because if people start using it for such things as an "is positive integer" check (Which, to be fair, they should not be doing. Nobody should be constructing 2 new javascript functions every time they want to check if an object is a positive integer. But apparently they are...), then it could easily make its way into somebody’s inner loops.
The end result of this general culture (not picking on anyone in particular) is that trivial Javascript programs end up having their performance tank, because every tiny bit of useful real work gets weighed down by two orders of magnitude of incidental busywork creating and tearing down abstractions built on towers of other abstractions. Because CPUs aren’t very well optimized for this kind of workload, which involves lots of branching and cache misses, it’s worse still, probably more like 4–5 orders of magnitude slower than optimized code written in a low-level language.
Ultimately every idle facebook page and gmail page and news article in some background browser tab ends up sucking up noticeable amounts of my laptop’s CPU time and battery, even when they’re doing nothing at all for me as the user.
When people ask why Javascript is terrible, show them this.
If you don't want to write modules this way, don't. Nothing about javascript requires that you even read articles about modules that you don't want to use. Or read articles and then follow up by posting on message boards about articles about modules you aren't going to use.
Bang some code out instead. Your opinion of javascript is about as valuable as the opinion of the person who wrote the modules.
There's no such thing as "don't use it if you don't like it!" in this case.
Javascript is a completely broken, shameful system that rightfully deserves all the scorn it's gotten. It really is the worst.
(note that i actually write Python & VanillaJS, and avoid Node packages entirely for now.)
If you're checking a value to see if it's set by testing for null/undefined, you're doing it wrong.
This is good advice for any language, not just JS.
Besides, using null as the default for an undefined value is a mistake. Null should indicate a non-value not the absence of a value. Maybe one day the static OOP languages will die so devs have a chance to learn the difference.
All these checks seem annoying but we've all seen the bar chart of when you discover the bug vs. the cost of that bug. The compiler finding the bug is cheap. Haskell is a cheap language to program in, compared to JS!
* I forgot there is no compiler in JS but let's say the linter for argument sake.
Null isn't used in JS to mark undefined variables, that's what 'undefined' is for. Unlike static/OOP languages, null is specifically reserved for use cases where a null value is necessary. Which was the point of my comment.
If you try to access an undefined variable in 'strict mode' it throws an undefined runtime error.
JSLint does, in fact, check for undefined variables and/or globals that are accessed before they're defined in the local source (ie unspecified external globals).
So... There's that...
-----
Did you happen to notice how I didn't even remotely mention Haskell or anything related to functional programming but, please, I'd love to hear for the thousandth time how Haskell's purity is going to cure the world of all it's ills. As if the Haskell community hasn't been over-promising and failing to deliver production-ready applications for the past decade.
Unlike Haskell, JS source can be actively linted while you write it rather than requiring a separate compile/build step.
With ES6 on the client-side, modules can be hot reloaded as changes are made for instant feedback. The obvious downside being, fewer coffe breaks that can blamed on the compiler.
Have fun implementing trampolines once Haskell's 'recursion all the things' default approach to lazy-loading inevitably overflows the stack and crashes your app in production. Static type checking can't save you from logic errors. See also 'memory profiling' the phrase that shall not be uttered.
Source: http://steve-yegge.blogspot.com/2010/12/haskell-researchers-...
I am genuinely interested in your point about trampolines and can you give an example? In Haskell foldl' for example allows you to process a big finite list quite efficiently without trampolines but yes the name of the function is a bit sucky I admit.
It's funny how you can criticize trampolines but then imperative code like this:
var sum = 0;
for( var i = 0; i < elmt.length; i++ ){
sum += parseInt( elmt[i], 10 ); //don't forget to add the base
}
var avg = sum/elmt.length;
document.write( "The sum of all the elements is: " + sum + " The average is: " + avg );
https://github.com/sindresorhus/ama/issues/10#issuecomment-1...
TL;DR: Small modules are easy to reason about, and encourage code reuse and sharing across the entire community. This allows these small modules to get a tremendous amount of real world testing under all sorts of use cases, which can uncover many corner cases that an alternative naive inlined solution would never have covered (until it shows up as a bug in production). The entire community benefits from the collective testing and improvements made to these modules.
I also wanted to add that widespread use of these small modules over inlining everything makes the new module-level tree-shaking algorithms (that have been gaining traction since the advent of ES6 modules) much more effective in reducing overall code size, which is an important consideration in production web applications.
Strictly speaking, you're definitely right. Each dependency is an additional point of failure, but so is each additional line of code you inline.
The benefits of these small modules is that they're very thoroughly tested by the entire community. I'd say for most cases, they will be much more robust than any solution an individual developer can spontaneously whip up.
Of course, these modules don't exist in a vacuum, and infrastructure failures such as this one do pose another point of failure you wouldn't have with inlining code, but I think in this particular case it had more to do with the failure of npm to adhere to package management best practices to not include an unpublish feature.
If someone can't make the right trade-off regarding "should I import a module called "isArray" or "is-positive-integer", then they should not be programming...
Dynamic typing and diversity of devices and virtual machines mean the ability to simple tell if something is an Array could be multiple lines of code that could take a considerable amount of time to test.
Delegating this to the community is arguably the only sane choice, however strange that may be to someone come from another environment.
There's probably not any overhead in bundling, though. Not in speed or memory, at least. The overhead is in the complexity: the programmer now has one more place to check for faults, and debugging stack traces now point to a bundled version of code instead of the original sources.
The case where none of this is true is when require() ends up being called thousands of times. Startup time will be correspondingly slow, and bundling is the cure. But that should only be done as a last resort, not a preemptive measure.
https://news.ycombinator.com/item?id=11349606
Basically, how does tree shaking deal with dynamic inclusions? Are dynamic inclusions simply not allowed? But in that case, what about eval? Is eval just not allowed to import anything?
I've been reading posts like these, but they are pretty unsatisfying regarding technical detail: https://medium.com/@Rich_Harris/tree-shaking-versus-dead-cod...
Maybe someone else was wondering the same thing, so I decided to post it here before wandering off to the ES6 spec to figure it out.
Correct. One of the motivating factors for ES6 modules was to create a format that can be statically analyzed and optimized.
> Is eval just not allowed to import anything?
Correct.
See this excellent writeup for more details regarding ES6 modules: http://www.2ality.com/2014/09/es6-modules-final.html
Discoverability though is so poor that most of those modules are most likely just used by the author and the author's co-workers.
If a typical npm user writes hundreds of packages, how the hell am I supposed to make use of them, when I can't even find them? Npm's search is horrendous, and is far from useful when trying to get to "the best/most supported module that does X" (assuming that random programmers rely on popularity to make their choice, which in itself is another problem...).
Would I write tests for my own leftpad implementation if I were not farming it out to a dependency? Its possible. More likely I would want to understand the intent behind using leftpad in my application or library, and have test for "formatTicket" or whatever it is that I'm actually doing. But for all the talk about tiny modules that cover surprisingly tricky edge cases, this is not one of them.
Why search through millions of unknown packages when a standard library would have it all right there?
left-pad was released under WTFPL, so in this particular case there'd be no legal barriers to it. (And I'd assume that, for any libraries with a restrictive enough license, it wouldn't be a hard sell on TC39's part -- if they put out an announcement that they were going to do that, I'd go panning for street cred, and I wouldn't be the only one.)
An alternative could be to pull all this stuff together into one big bucket of Legos and package it with a code analysis tool at the beginning of your build process to strip out everything you don't need from the bucket... but I'd guess that's either already been done or a stupid idea for reasons I don't know yet.
Now having a metalibrary that depends on a curated collection of popular/useful utilities is probably a good idea - but isn't that exactly what many of the libraries that broke due to depending on left-pad were?
That way I can get a curated list of great functions I can trust.
Essentially a standard library for all intents and purposes.
We can't exactly search the code to find out what it does, otherwise you'd basically be reading all the code to discover it's true behavior, which negates the usefulness of modules in the first place...
Any search will have to rely on developer-made documentation, and/or meta data. This is great in theory, but documentation is rarely well maintained, and/or someone changes the code but neglects to update the documentation.
This leaves us with the situation we have today. A search that somewhat works kindof, and mostly you rely on googling the feature you need and looking for what seems to be the most popular choice.
I'm not sure how this situation can be made better, especially if we continue down a path of having all these "one-liner" libraries/modules that anyone can make and stick out there into the unsuspecting world. When I need a square root function, and my search returns 800 one-liner modules, how am I supposed to pick one that I know is reasonably well done, and does what it says it will do, without looking under the hood - you'll end up just picking whatever seems to be popular...
That would be very cool, but I'm not sure how much easier it would actually turn out to be. Also to do anything useful you'd probably have to restrict the language to be non-Turing-complete.
(Same reason the pseudo-Idris language would have to be non-Turing-complete)
This is why types are better than textual documentation - the system enforces that they're kept up to date.
Intuitively, this thread wouldn't exist if your assertion were correct.
Most of the small libs I've shared are mainly sharing unit tests.
Here's a function I've shared on npm:
var kind = function(item) {
var getPrototype = function(item) {
return Object.prototype.toString.call(item).slice(8, -1);
};
var kind, Undefined;
if (item === null ) {
kind = 'null';
} else {
if ( item === Undefined ) {
kind = 'undefined';
} else {
var prototype = getPrototype(item);
if ( ( prototype === 'Number' ) && isNaN(item) ) {
kind = 'NaN';
} else {
kind = prototype;
}
}
}
return kind;
};
suite('kind', function(){
test('shows number-like things as numbers', function(){
assert(avkind(37) === 'Number');
assert(avkind(3.14) === 'Number');
assert(avkind(Math.LN2) === 'Number');
assert(avkind(Infinity) === 'Number');
assert(avkind(Number(1)) === 'Number');
assert(avkind(new Number(1)) === 'Number');
});
test('shows NaN as NaN', function(){
assert(avkind(NaN) === 'NaN');
});
test('Shows strings as strings', function(){
assert(avkind('') === 'String');
assert(avkind('bla') === 'String');
assert(avkind(String("abc")) === 'String');
assert(avkind(new String("abc")) === 'String');
});
test('shows strings accurately', function(){
assert(avkind(true) === 'Boolean');
assert(avkind(false) === 'Boolean');
assert(avkind(new Boolean(true)) === 'Boolean');
});
test('shows arrays accurately', function(){
assert(avkind([1, 2, 4]) === 'Array');
assert(avkind(new Array(1, 2, 3)) === 'Array');
});
test('shows objects accurately', function(){
assert(avkind({a:1}) === 'Object');
assert(avkind(new Object()) === 'Object');
});
test('shows dates accurately', function(){
assert(avkind(new Date()) === 'Date');
});
test('loves Functions too', function(){
assert(avkind(function(){}) === 'Function');
assert(avkind(new Function("console.log(arguments)")) === 'Function');
assert(avkind(Math.sin) === 'Function');
});
test('shows undefined accurately', function(){
assert(avkind(undefined) === 'undefined');
});
test('shows null accurately', function(){
assert(avkind(null) === 'null');
});
});But naturally, with any code reuse there's a benefit and a cost to instance of internal or external reuse.
The Benefits for external reuse include ideal reliability as your describe as well as not having to create the code. The costs for external reuse include having your code tied to not just an external object but also the individuals and organizations creating that object.
I think that means that unless someone takes their hundreds of modules from the same person or organization and is capable of monitoring that person, that someone is incorporating a layer of risk to their code that they don't anticipate at all.
There is always a balance between getting some extra productivity by using somebody else's code, or getting the knowledge, expertise and control of writing it yourself. Far too often I see young programmer's choose something off the shelf so that they don't have to understand it. This is a recipe for disaster -- for your project, for your team and for yourself.
IMHO, as programmers, we should err on the side of writing code. Often it is obvious that you should use standard libraries, or a particular framework or or another reuse library. But where it is not really obvious, I would start with writing your own code and see where it gets you. You can always replace it later if it doesn't work out. I think you will find that you and your team will be miles ahead if you take this approach.
in python: https://docs.python.org/3.3/library/stdtypes.html#str.ljust
I don't want to claim that it would be a directly calculated career move, more like starting a blog: you admire a good blog, you want to be more like the blogger, you start your own one. On the dim chance that it will become both good and not abandoned after the third post. Nanomodules can be just like that, the air guitar of would-be rockstar coders. This is certainly not the only reason for their existence (and even the air-guitar aspect has a positive: low barrier of entry that might lead to more serious contribution), but the discussion would be incomplete if it was ignored.
Step 1: Create a culture in which "having open source contributions" is a requirement to entering said culture.
Step 2: Remove all friction from introducing open source contributions into the culture.
Step 3: Watch the Cambrian explosion.
Step 4: (Two years later) Point to the Cambrian collapse and how the new hot thing will solve everything.
I don't know what sort of shit show Step 4 will turn into, but it will also definitely be the result of folks taking a simple and good rule of thumb (this time, it won't be "small and composable"), and implementing it without ever stopping to think what problems it solves and doesn't solve.
Step 5: a mechanism for curated metamodules
This would not have changed much for the current situation, but it might help cultivating a "too important to play it fast and loose" mindset that would.
It's about unforeseen consequences of a solution to a problem. That's what I mean by "I don't know what it will look like", because it will definitely solve the problem this thread is dissecting, and the problem it introduces will be a whole new vector.
But its selling point (the way it solves our current problem) will be a simple idea that is taken to its logical extreme by people who don't think critically, and then it will be time to solve that problem.
That is, I see the underlying recurring problem as stemming from the cultural forces –– how we educate people, how we make ourselves hireable –– that enable even very smart people to be shortsighted and immune to learning from the past.
On the contrary, I would certainly mark "maintains moderately popular open source project" as a calculated career move, just like how prostitutes would dress up right for the cliente. Sorry for the crude example but I am not placing the blame on the devs here but the absurd notion of employers considering a GitHub repo the 'in' thing before actual capability. Candidate - Brah I got 10 of those packages with 10mil downloads in total but I can't code for shit. Hiring manager - Since we are suckers who look at stats, you're hired!
The problem with this nanomodules approach is that it results in complacent and naive developers that eschew the basics and just publish random crap. Anything can make it to be a NPM package and be judged by...popularity? Since when is code turning into pop music :)?
You see, the whole Fizz Buzz is nothing more than a prop to allow me to find out what this candidate can actually do. Anyone can include a package and critical thinking differentiate smart ones from the pack. Heh, if there is a package for any and everything, most devs would be flipping burgers instead of doing original work.
Believe me, most problems you think of as original, aren't original. It's not about solving problems either, it's about repacking the solutions into something to sell, or making it popular enough so you can make money off it.
Yes they are, in the same way that a book in which every page consists of a single word is easier to understand than one with more content per page.
By focusing on the small-scale complexity to such an extreme, you've managed to make the whole system much harder to understand, and understanding the big picture is vital to things like debugging and making systems which are efficient overall.
IMHO this hyperabstraction and hypermodularisation (I just made these terms up, but I think they should be used more) is a symptom of a community that has mainly abandoned real thought and replaced it with dogmatic cargo-cult adherence to "best practices" which they think will somehow magically make their software awesome if taken to extremes. It's easy to see how advice like "keep functions short" and "don't implement anything yourself" could lead to such absurdity when taken to their logical conclusions. The same mentality with "more OOP is better" is what lead to Enterprise Java.
Related article that explains this phenomenon in more detail: http://countercomplex.blogspot.ca/2014/08/the-resource-leak-...
A more apt metaphor would be separating a book into many small paragraphs that each serves a single purpose makes the book easier to understand. Regardless, the book metaphor misses a lot of the nuance.
Of course taking the approach to an extreme would be detrimental. However, a vast majority of these small modules that are actually widely used consist of functions that may seem trivial at first sight, but actually contain a lot of corner cases and special considerations that a naive inline implementation could miss.
Sure, there are also plenty of trivial one-line modules on npm that don't fit into a such a description, but those are simply side effects of the unprecedented popularity of the platform and its completely open nature, and shouldn't be used to infer any kind of general trend towards "hypermodularisation" because very few reasonable developers would ever import them into their projects.
No, that would not be an apt metaphor for the problem he is describing.
> Regardless, the book metaphor misses a lot of the nuance.
Not if you are trying to understand the point he is trying to make.
Is there any cost to creating modules, uploading them to npm and using them in other projects ? Clearly there is, as is shown by the world-wide breakage from yesterday. This is probably only one of such failure modes.
The point is, breaking everything into micro-modules can have benefits and costs. Ultimately, it is an engineering trade-off.
Now, it is possible that npm modules are at the right level of abstraction, even though other module systems in the world are not this granular. If this is due to the special nature of JavaScript then the point must be argued from that perspective.
That's because I don't agree that the problem he is describing exists, or at least not to the degree he's describing, which was full of hyperbole.
> Is there any cost to creating modules, uploading them to npm and using them in other projects ? Clearly there is, as is shown by the world-wide breakage from yesterday. This is probably only one of such failure modes.
This was a failure on npm's side by including a functionality that allows users to trivially remove packages from a package management system that is used by hundreds of other packages, something that most major package management systems have decided was a bad idea.
> The point is, breaking everything into micro-modules can have benefits and costs. Ultimately, it is an engineering trade-off.
Agreed. And I don't think nearly as many people are erring on the extreme side of this tradeoff to the degree that he's describing.
That is emphatically not the problem. The author of those modules could have just as easily modified the code instead of deleting it:
function leftpad(str, len, ch) {
// NPM won't let me delete modules, so here you go
return "";
}
The problem is the dependency, not the precise mechanism by which that dependency screws you when it goes bad.
Because it is a strawman.
Other languages handle this with compilers, may be strongly typed, and/or have features that don't allow nulls. Javascript doesn't exactly have that luxury. So maybe it makes sense in Javascript, where it wouldn't in other languages (though one could argue this is a flaw in the language design).
edit: to be clear I'm not really defending the practice, but trying to give a little perspective.
#include <math.h> and "-lm"
with
#include <sin.h> #include <cos.h> #include <tan.h> #include <sinh.h> …
and "-lsin -lcos -ltan -lsinh …"
Which is nuts no matter what language you are coding in.
Key quote from brilliant word-talking-guy Henry Fowler: "Sarcasm does not necessarily involve irony [but it often does] ... The essence of sarcasm is the intention of giving pain by (ironical or other) bitter words."
So, we circle back to Node.js by way of pain and bitterness. Sounds about right.
With javascript however, if you import a hypothetical "math.js" instead of just "sin.js", "cos.js" or "tan.js", then you'll need to download and evaluate a whole bunch of javascript that you might not need.
I'm not defending javascript, because I dislike it and avoid it where possible, but I can understand some of the reasons why short, specific modules came about.
-flto, on the other hand, allows to reverse this process to allow interprocedural optimization across different translation units.
@majewsky: what you're describing is more akin to "-ffunction-sections" and "-gc-sections" which will split ewach function into a separate section and garbage-collect those sections.
Not quite. When you link to a static library (.a) with lots of .o object files in it, only those object files will be linked that are actually used by your program.
I first learned about this when I looked at the source of dietlibc, and wondered about every function being in a separate file. That enables the aforementioned trivial optimization, even without -flto.
I've written worse - at least those cover multiple variations each (or overloads in C++ for float/double/std::complex?/...)
While I'm not a fan of the enforced java route of 1 file = 1 class, I do trend towards 1 file ~ 1 (main, public) thing - which might be a single function with no overloads. #include <string_left_pad.h>? Better than #include <everything.h>, which I see far too often...
I don't have to figure out which grouping of things my coworkers decided to throw something into if I can just #include by name - whereas aggregation headers more often than not will trigger a full project file-system search.
Unnecessary #include s don't accumulate nearly so much when you don't have to audit 100 functions to see if any of them use features from said header.
I don't trigger a rebuild of everything when the only things that #include stuff actually need it.
Lots of benefits!
> and "-lsin -lcos -ltan -lsinh …"
Or sin.o, cos.o, tan.o, sinh.o... which suddenly sounds a lot more reasonable. I wouldn't bat an eye at string_left_pad.o either.
Sure, I want to use tools to simplify the use of those micro-modules by aggregating them into libraries for convenience. But that's not a knock against micro-modules - just their packaging and tooling. Rewind time back far enough and I would've been manually specifying .o files on the command line and kvetching too...
For example this is totally legit:
// ClassA.java
public class ClassA {
public static class ClassA_Inner_Public_Static {
}
public class ClassA_Inner_Public {
}
}
// ClassB.java
public class ClassB {
ClassA classa = new ClassA();
ClassA_Inner_Public classA_Inner_Public = new ClassA().new ClassA_Inner_Public();
ClassA_Inner_Public_Static classA_Inner_Public_Static = new ClassA_Inner_Public_Static();
}1) moving it into a separate file, for the simplest two value one liner
or
2) making it interior to the other class, with all the verbose lengthy names (and semantic implications) that this might entail.
I want clear and concise names, so I lean towards #1. And if an enumeration gets more shared between multiple classes, or collects enough values and comments to merit it, sure, I'll move it into it's own file. But forcing me to shove it in a separate file before I've even decided on the best name (meaning I need to rename the file repeatedly as well), and before I'm even certain I'll keep that specific enumeration, is just meaningless friction.
Why don't you want the whole module? Because it's bloat? Surely it's far less bloat than one submodule per function?
Every line of code that isn't used is bytes you're sending to every user unnecessarily
> Surely it's far less bloat than one submodule per function?
How is that bloat? It's the same LoC whether it's 1000 modules or 1 after it's been compiled/minified
I'm not arguing that there shouldn't be a math module but that you shouldn't split it up into such (imho) crazy small parts.
Closure Compiler can remove any unused functions.
It shouldn't be (impactful) bloat or overhead in the build system or dependency management. That stuff should just handle it.
Most of the time you don't care and can just declare the dependency on math and let the closure compiler handle it. But sometimes it might be really important that a project only used cos and not sinh, in which case you want to enforce that.
That depends; in the case of mathematics libraries they have to be bundled together because of functional dependencies. It's either that or ignoring DRY principles -- which if you do that, you're ignoring the entire point of what breaking things into submodules is intended to do.
It's easier to keep an eye on a small number of trusted parties than 200 random strangers.
You thought this SNAFU was bad...
Someone has to do this manually?! If the package is not popular, no one cares? What happens if I send them an email and provide the same package with the same API (not trademarked and MIT licensed) but break it a bit on every update?
No one knows.
when was the last time you rechecked the entire source of a package you depend on after updating it?
In practice, many, if not most, of these one-line modules don't even work correctly, and it is difficult to get people to care to collaborate on fixing them instead of just replacing them with a new one-line module that works slightly better as the concept of complex collaboration to perfect one line of code is so foreign to a generation of developers who would rather just throw code to the world and never look back (an issue made all the worse as tiny bugs in these one-liners can almost seem dangerous to fix when you aren't really sure how they are being used: maybe it actually is better to not fix anything and just have people rely on the replacement better module :/).
Words and phrases have lost their meaning nowadays.
By the JS community's standards if I wanted code to capitalize the 3rd or sometimes 4th letters of a string, they would be 2 different npm modules.
Maybe they didn't forget how to program, as the article implies; maybe they never knew how to program in the first place.
> By the JS community's standards if I wanted code to capitalize the 3rd or sometimes 4th letters of a string, they would be 2 different npm modules.
If there was, it would have been published and people would be depending on it. Instead, people use the stdlib first, then big lego bricks like underscore, then (when they don't want the bulk of big lego bricks) smaller lego bricks. A single capitaliseMap() might be in there but it's a much more specific case than padding.
If you had years of experience in a wide array of technologies and languages, you'd be aware of that which is, and has been, obvious to the rest of us for the past 2-3 decades.
Nothing about function length determines utility. There are many one lines that match many people's needs repeatedly.
> When you have 100 such modules which don't quite work
This is worrying. How are you picking the modules you use? 100x popular modules from npm - with git repos, unit tests, READMEs, and hundreds of other users - beat 100x functions implemented in-house for NIH (or more likely, 100x functions copy pasted from Stack Overflow).
> If you had years of experience in a wide array of technologies and languages
Please don't assume things about other people. It's very rude, and it makes you look bad when you're wrong.
At least there are JS programmers that seem aware of it and agree that this has to change. So there's still hope I guess.
I completely agree. I still can't understand why the language that we depend on for increasingly large-scale front-end work still works like this:
[1, 10, 2, 21].sort() // returns: [1, 10, 2, 21]
It's 2016, software engineering is purported to be a profession, and we still have the worst tools imaginable.
.sort(sorts.numerical)
Using npms 'sorts' module
Diseases turtles all the way down.
Where? I've only seen the exact opposite: folk who don't believe in package management write their own naive 'reinvent the wheel functions' with those issues. Nobody I can see is quoting actual npm modules.
But to be explicit about the history: version one[0] had dependencies. Version two[1] had no dependencies, and a comment on github suggests that it might be wrong but I can't immediately see how. Version three[2] is a different implementation, and adds another export.
[0] https://github.com/tjmehta/is-positive-integer/commit/3003f9...
[1] https://github.com/tjmehta/is-positive-integer/commit/b47e90... - it should return false for 0, so I'm not sure what the commenter is getting at
[2] https://github.com/tjmehta/is-positive-integer/commit/3cdfe1...
It's really nice how we once again depend on "I think some guys do it like this" for performance. What happened to standards?
What's next? SEO for code?
What's funny to me is that leftpad(null, 6, ' ') outputs " null".
Here is some preformatted text.
null
"null"
left-pad isn't even slightly abstract.
The hyperabstraction is in how even tiny functions like this, isPositive, isArray, etc. are being abstracted and turned into individual modules.
But abstraction? I don't see how that word is connected to what's happening. Maybe you can explain it.
Frankly some of the comments I'm seeing really do reinforce his point: people really don't know how to program. I suspect this is it the flipside of Javascript being popular and accessible - people can churn out products without really knowing what they are doing...
Well put. I've noticed a curious blind spot in how people account for complexity: we count the code that 'does' things much more than the code that glues those things together. This distorts our thinking about system complexity. A cost/benefit analysis that doesn't consider all the costs isn't worth much.
An example is when people factor complex code into many small functions, then say it's much simpler because smaller functions are easier to understand. In fact this may or may not be true, and often isn't. To get a good answer you must consider the complexity you've added—in this case, that of the new function declarations and that of the calls to them—not just the complexity you've removed. But it's hard to consider what you don't see. Why is it so easy not to see things like this? I think it's our ideas about programming, especially our unquestioned assumptions.
The complexity of glue code goes from being overlooked to positively invisible when it gets moved into things like configuration files. Those are no longer seen as part of the system at all. But of course they should be.
https://youtu.be/l1Efy4RB_kw?t=343
He refactored a Java application into 1400 classes!
How does that even remotely applies to the "is positive integer" test, and even more so to "it's positive" and "it's integer"?
What's next? is-bigger-than-5? word-starts-with-capital-letter? add-1-to-a-number?
This seems to be a very common example in educational materials about how to create a function in some programming language. Perhaps they should all be prefaced with "NOTE: you should not actually make such a function, because its triviality means that every time you use it you will be increasing the complexity of your code", or a longer discussion on when abstraction in general is appropriate and when it is not.
Ditto for any other abstraction --- I've found there's too much of an "abstraction is good so use it whenever you can" theme in a lot of programming/software design books and not enough "why abstraction can be bad" sort of discussion, which then leads to the "you can never have too much" mentality and the resultant explosion of monstrous complexity. Apparently, trusting the reader to think about the negatives and exercise discretion didn't work so well...
That's the whole point. If you use is-positive-integer, you won't have to think about the negatives!
module.exports = exec("===");
module.exports = bigger(a, b){return a > b};
module.exports = () => return "than";
module.exports = (a) => a.search('ABCD...');
.. or something like that. These are all plugins of course.
Of course we have a number of packages that are truly trivial and should probably be inlined. Npm is a completely open platform. Anyone can upload any module they feel like. That doesn't mean anyone else will feel the need to use them.
I think you'll find that the vast majority of small modules that are widely used are ones that also cover obscure corner cases and do benefit from widespread use and testing.
We're doing a GSoC for efficient templating of real number though.
Well played.
You wouldn't write: "I saw Five of my friends last night."
Having pre-written modules that account for all edge cases that have been battle-tested by hundreds, thousands, or even millions of users does have merit.
The main problem here is with the Javascript language, which makes code evolve to things like 'is-positive-integer'.
The issue I have with this is readability. Type '>' and I know exactly what it does, I know what implicit conversions are involved, and how it would react to a null object. Type 'isPositiveInteger' and I need to check. I can not read your code fluently anymore.
I feel like you haven't spent enough time with Unicode.
You're welcome!
edit: fortunately there's an npm modules you can included to fix it...
Why not both? If you just want one part of a module, you can just import the dependency directly, if you want a whole core lib, you can do that.
Some people really like underscore and import that. I use ES6/7 functions for the most part, but underscore has some useful things I can import on their own.
startsWithCapitalLetter("!") == true
> isCapital("שלום");
true
> isCapital("1");
true
> isCapital('\uD83D\uDE00'); // Smiling emoji
true
> isCapital(\u279B); // Right-facing arrow
true
> isCapital("\u061Casd"); //Bidirectional control character
true
> isCapital(" ");
true
Actually, it just means that if you're wondering "if this word starts with a capital", you're asking a wrong question. Instead, you should be asking "if this word is a valid generic name", or "is this word a start of a new sentence", and implement these semantic queries in a language-specific way.
Adding another module with a method does not fix the original problem, it just creates more problems for other people to solve.
i++
Someone might do something like ~~n to truncate a float. That works fine until you go above 32 bits. Math.trunc should be used instead. Someone might do something like n >>> 0 === n, and using any bitwise operators will always bake in the assumption that the number is 32 bits. Do you treat negative 0 as a negative integer? Detecting negative 0 is hairy. So, to avoid bad patterns, it makes sense.
For is-bigger-than-5(x), x > 5 is not hairy. For add-1-to-a-number(n), n + 1 is not hairy.
For word-starts-with-capital-letter(word)? That one is actually pretty hairy. There are programmers that would write a regular expression /^[A-Z]/, or check the charCode of the first character being within the A-Z range, amongst other solutions. An appropriate solution, however, would be (typeof word == "string" && word.length && word[0] == word[0].toUpperCase() && word[0].toUpperCase() != word[0].toLowerCase()), because the toUpperCase and toLowerCase methods are unicode-aware, and presently you can't access unicode properties using Javascript's flavor of regular expressions.
Yes, at some point the complexity cost of gluing together the standard library functions to do something becomes greater than the lookup cost of finding a function that does what I want; but I am saying that adding more functions is not costless.
This is madness.
[1] https://github.com/sindresorhus/float-equal/blob/master/inde...
and no, the edge cases argument does not apply to either of them. Wow.
If it's a popular issue, lots of people had the same issue, many will be nice enough to add their edge cases and make the answer better, most will not. Same goes for contributing to a package.
With a package you would be able to update when someone adds an edge case, but it might break your existent code and that edge case may be something that is not particular to your system.
If you don't want to get too deep in the issue, you can copy paste from SO, just the same you can just add a package.
If you want to understand the problem, you can read the answers, comments, etc. With the package you rely on reading code, I don't know how well those small packages are documented but I wouldn't count on it.
The only arguments that stands are code reuse and testability. But code reuse at the cost of the complexity the dependencies add, which IMO is not worth the time it'll take you to copy and paste some code from SO. Testability is cool but with an endless spiral of dependencies that quite often use one or more of the different (task|package|build) (managers|tools) that the node ecosystem has, I find it hard to justify adding a dependency for something trivial.
How do I systematically make sure that I have the latest version of every stackoverflow code snippet? If it's a new post, it may not have all the edge cases fixed yet. So now I have to check back on each of the X number of snippets I've copied.
In the npm approach, I can easily tell if there's a new version. For prod, I can lock to a specific version, but in my test environment, I can use ^ to get newer versions and test those before I put them in production.
If the edge case of new version of a package breaks my code, I've learned that I'm missing a unit test. Plus, the question isn't whether this bad thing might happen on occasion, the question is whether this approach is, on balance, superior to cutting and pasting random code snippets into my code. I think the downside of the npm approach is less than the downside of the copypasting from stackoverflow approach.
And every moderately useful npm package I've looked at has very good to great documenation.
https://github.com/sindresorhus/ama/issues/10#issuecomment-1....
If anything looking at sindresorhus's activity feed: (https://github.com/sindresorhus) perfectly supports the author's point. Maybe some people have so little to do that they can author or find a relevant package for every single ~10 line function they need to use in their code and then spend countless commits bumping project-versions and updating package.json files. I have no idea how they get work done though..
"...LOC is pretty much irrelevant. It doesn't matter if the module is one line or hundreds. It's all about containing complexity. Think of node modules as lego blocks. You don't necessarily care about the details of how it's made. All you need to know is how to use the lego blocks to build your lego castle. By making small focused modules you can easily build large complex systems without having to know every single detail of how everything works..."
By LOC he's referencing 'ye ole Lines of Code paradigm, and trying to make the point that in the end it just doesn't measure up to the prime directive of "containing complexity".
... and that's where I beg to differ.
What I think is being completely overlooked here is the net cost of abstracting away all that complexity... It's performance.
Every extra line of code, extraneous function call, unnecessary abstraction ( I'm looking at you promise ), every lazy inclusion of an entire library for the use of a shortcut to document.getElementById -- these all add unnecessary costs from the network down to the cpu.
And who pays these costs?
The end users of your slow, memory hogging, less-than-ideal set of instructions to the cpu that you call an application... but hey, it's easier for you, so there's that.
promise.then(function(){}).then(function(){})
async.waterfall([function(){}, function(){}])
Which was: throwing away the entire language's compositional features, including the concept of input arguments and return values, results with... poor compositionality, of course.
It seems you're defining callback hell as 'whatever promises solves' rather than it's common definition of over-nesting.
Node style callbacks are a prime example of how in the quest for "simplicity" and hate for abstractions its easy to build a monster / frankenstein like caolan's async. Node callbacks were a self-contradicting philosophy: it abandoned all thats already in the language in the quest to avoid an abstraction that was not already in the language :)
That said I see how the inconsistency you're talking about with how expressions return values with callbacks and you've certainly put the point very well.
So promises are way more composable than callback based solutions.
async is pretty terrible though, you cannot pass the results from one execution to another, i.e. passing results from one function in async.series to another, which results in developers tending to pollute variable scope by defining above and filling it in inside each callback. This prevents the ability to write clean isolated testable functions.
Little-known fact: the *y in old signs is actually a 'þ', which is an Old English letter named 'thorn' and pronounced like modern 'th.' Thus, the old books & signs were really just saying, 'the.' Incidentally, þ is still used in modern Icelandic.
Completely off-topic, but I þought you might be interested.
edit: and I double checked, because the internet :)
http://www.etymonline.com/index.php?term=ye
it's true! that's awesome.
Rather than copy paste someone else's unmaintained thing and handle all the bugs, or write their own code and unit tests, they use the same library as a few thousand other people?
Small modules are easy to reason about
If you had written (x > 0) I would have known immediately, it also wouldn't be doing the same thing as is-positive-integer(x) but how many calls to is-positive-integer are actually correct in all the corner cases that is-positive-integer covers?
And then there's the other problem with dependencies: you are trusting some unknown internet person to not push a minor version that breaks your build because you and Dr. is-positive-integer had different definitions of 'backwards compatibility'.
JS does not have an Integer type, so (x > 0) does not work
PS. never mind that the implementation of is-positive-integer is extremely inefficient.
function isPositiveInteger(x) {
return x > 0 && Number.isInteger(x);
}
That's the root of all the problems, though, isn't it? JavaScript is just a terrible language.
Actually, that's not really fair. It's a great little language for writing late-90s Dynamic HTML, maybe just a little too powerful. And that additional power enables people to build cathedrals of cruft in order to Get Things Done.
I don't like Scheme (Lisp is IMHO better for real-world, non-academic work), but I still wonder how much things would have been if Eich had been allowed (or had chosen) to implement a small Scheme instead of JavaScript. The world of web software engineering would be so profoundly better.
Maybe WebAssembly will save us all.
But I don't think an initial choice of implementing a Scheme would have helped. The idea of +0 vs 0 vs -0 could just have easily happened in a Scheme, same too for the reliance on stringy types. Those are symptoms of a rushed design and an unwillingness to accept the temporary sharper pain of invalidating existing bad code to produce a new better standard (the exact same tendency - to dig in deeper rather than risk the pain of turning back - is literally how spies managed to convince people to commit treason).
Then of course there's also the great risk that, just like Scheme not-in-the-browser, Scheme-in-the-browser might never have widely caught on.
Yeah, there's even a Common Lisp which compiles to JavaScript …
> The idea of +0 vs 0 vs -0 could just have easily happened in a Scheme, same too for the reliance on stringy types.
I don't necessarily know about these specific examples: the Scheme standards have been quite clear about their numeric tower and equality standards.
I think your general point about the hackiness which was the web in the 90s, and the unwillingness to break stuff by fixing things holds, though. And of course it wasn't just the web: I recall that the Makefile syntax was realised to be a mistake weeks into its lifetime, but they didn't want to fix it for fear of inconveniencing a dozen users (details fuzzy).
> Then of course there's also the great risk that, just like Scheme not-in-the-browser, Scheme-in-the-browser might never have widely caught on.
I dunno — would a prototypal language have ever caught on were it not for the fact that JavaScript is deployed everywhere? I can imagine a world where everyone just used it, because it was what there was to use.
And honestly, as much as I dislike Scheme, it would have fit in _really_ well with JavaScript's original use case and feel. And if the world had had a chance to get used to a sane, homomorphic programming language then maybe it might have graduated to a mature, industrial-grade, sane, homomorphic language.
But alas it never happened.
return Number.isInteger(x) && x > 0;
Or maybe you don’t just want safe integers, maybe you want `x >>> 0 === x`, or `x > 0 && (x | 0) === x`, or…. This is what I meant about JavaScript being confused about the definition of “integer”.
Don't even get me started on negative zero.
I didn't know the >>> operator converts to 32-bit. How does it do that? Mask off the top bits? Or give you a NaN if it's bigger?
But (and someone please correct me if I'm wrong), at the end of the line it's still represented as a 64-bit float internally and accurately.
Too many modules do not necessarily become a good thing. They may appear to get rid of complexity but in reality, you will have to face the complexity some level above and in fact, the sheer number of small modules will most probably add more complexity of themselves.
Flip-side: it isn't easy to reason about a large and complicated graph of dependencies.
"It's all about containing complexity." - this completely ignores the complexity of maintaining dependencies. The more dependencies I have the more complicated my project is to maintain.
Dependencies are pieces of software managed by separate entities. They have bugs and need updates. It's hard to keep up to date.
When I update a piece of software I read the CHANGELOG, how am I expected to read the CHANGELOG for 1,000 packages?
Depending on a bigger package (handled by the same entities, who write one changelog, in the same form) is more straight forward.
I'm not saying this is wrong - but there's a balance here, and you must not ignore the complexity of increasing your number of dependencies. It does make things harder.
That's not a Lego block; that's an excuse.
isArray(arr)
toString.call(arr) == '[object Array]'
Anyone that has been around long enough has war stories about getting two relatively simple pieces of software working with each other. In my experience, integration problems are often the most difficult to deal with.
There's even some guy calling for a "micro-lodash". To me, as a Python engineer, lodash [1] is already a tiny utility library.
I guess it's also about the fact that JS is a pretty bad language. That you need a one-line `isArray` dependency to `toString.call(arr) == '[object Array]'` is crazy.
That said, lodash core is only 4KB, and lodash gives you the ability to build it with only the categories or exact functions you want, so I don't understand what the purpose of a "micro-lodash" would be.
Nope.
Not to mention the cognitive overhead of stopping programming, going to NPM, searching/finding/installing the module, then reading the documentation to understand its API. Isn't it simpler to `while (str.length < endLength) str = padChar + str;`? How can there be a bug in that "alternative naive inlined solution"? Either it works or it doesn't!
The derision is unwarranted, due to a failure in critical thinking from otherwise smart people.
Risk of indirectly hosing a project with N module owners providing dependencies: 1-((1-H)^N)
Let's say H is very small, like 0.05% of module owners being the type who'd hose their own packages.
3 module owners: 0.15% chance your own project gets hosed
30 module owners: 1.49% chance your own project gets hosed
300 module owners: 13.93% chance your own project gets hosed
Keep in mind it's not just your dependencies, but your entire dependency chain. And if you think a module owner might hose some modules but not others, maybe H is actually the number of modules in which case 300 starts getting pretty attainable.
Upshot:
Not everyone is trustworthy enough to hang your project on. The more packages you include, the more risk you incur. And the more module owners you include, definitely more risk.
The micromodule ecosystem is wonderful for all the reasons described, but it's terrible for optimizing against dependency risk.
Takeaways:
Host your own packages. That makes you the module owner for the purposes of your dependency chain.
If you're not going to do that, don't use modules dynamically from module owners you don't trust directly with the success of your project.
I love collaborative ecosystems, but some people suck and some perfectly non-sucky people make sucky decisions, at least from your perspective. The ecosystem has to accommodate that. Trust is great...in moderation.
I wanted to use a javascript tool that would make my life easier, and when I looked at the npm dependency tree it had 200+ dependencies in total.
If I used that javascript tool, I'd be trusting hundreds of strangers, lots of which had absolutely no clout in github (low number of stars, single contributor projects) with my stuff.
And not just them, I'd be trusting that no one steals their github credentials and commits something harmful (again, these projects are not very popular).
It doesn't help that npm doesn't (AFAIK) implement code signing for packages which at least would let me manage who I choose to trust
In all the debate about this, why is the trust-dependency-fuck-show not getting more attention?
Every dependency you take is another degree of trust in someone else not getting compromised then suddenly finding all sorts of horribleness making it into your production environment.
It beggars belief!
I move out modules so I can write really nice tests for the independent of the projects I am using them in. Also, I tend to write projects w/ 100% test coverage, breaking out utils allows me to test projects easier and faster.
Also note, the implementation of this module changed a few times today. With it being open source and having the collaboration of other engineers, we ended up with a very performant version, and discovered interesting quirks about "safe integers" in JS.
Isn't that why we just write functions? Turning simple functions into entire modules just adds an unnecessary level of abstraction that helps nobody.
It's like (micro)crowdsourcing (the smallest components of) the standard library that JavaScript never had.
Some bit of logic could go from being DRY in one project, to DRY in all of my projects, ... to eventually be DRY in all projects. It's globally DRY.
- Breaking out is-positive-integer hasn't reduced the number of paths to test. You have not gained anything, you've added overhead.
- 100% test coverage is rarely a good thing. It is required for safety critical areas like avionics. I can guarantee that your JS code is not making into any safety critical environment!
But it has: it's now tested, and you don't need to write tests for it next time you want to use it.
>100% test coverage is rarely a good thing
Not sure what your argument is here. Sure, it may not be helpful but are you saying that one should strive for less than 100% coverage, because "it's rarely good"?
Do you mean rarely useful, or actively harmful?
What happens when you have a "100% test coverage" requirement is that people don't think about the tests, they simply make tests to force the code down every path without thinking whether it was intended to operate like that.
For example if the is-positive-integer had a silly test for "if(value==23) return false", a requirement for "100% test coverage" would simply result in someone creating a test for that condition instead of considering if it was actually a fault.
100% test coverage != no faults.
What you have done by generating 100% test coverage is effectively 'frozen' your code and made it harder to implement any changes.
It's like following someone's car and congratulating the driver that he drove correctly, without considering if he reached the correct destination.
isPositive = (x) => +x === x && x > 0
This is especially true for one-liner modules in js, where any test at all might let you claim 100% statement coverage, without accounting for branches or loops within method calls.
As someone who spends 80% on the back end, I often get bit on the ass from JavaScripts quirks when I need to add some front end stuff.
This article touches on things that are wrong in the javascript culture. I always had this nagging feeling when working with NPM, this article brings it to light. For what it's worth I never felt this while writing Ruby, C# or Go.
It's the -culture- that needs to change here, not the tools.
Plus, if you think that's too small, write your own broader module that does a bunch of stuff. If people find it valuable, they'll use it. If they find it more valuable than a bunch of smaller modules, you'll get 10,000 downloads and they'll get 10.
The module you roundly ridicule has had 86 downloads in the last month, 53 of which were today (at the time of this writing). I imagine most of those 53 were after you posted. So that's 40 downloads in a month, as compared to the express framework which has had 5,653,990 downloads in the last month.
The wailing and gnashing of teeth over this module is ridiculous.
> I think that it's extraordinarily important that we in computer science keep fun in computing. When it started out, it was an awful lot of fun. Of course, the paying customers got shafted every now and then, and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful, error-free perfect use of these machines. I don't think we are. I think we're responsible for stretching them, setting them off in new directions, and keeping fun in the house. I hope the field of computer science never loses its sense of fun. Above all, I hope we don't become missionaries. Don't feel as if you're Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don't feel as if the key to successful computing is only in your hands. What's in your hands, I think and hope, is intelligence: the ability to see the machine as more than when you were first led up to it, that you can make it more.
Quoted in The Structure and Interpretation of Computer Programs by Hal Abelson, Gerald Jay Sussman and Julie Sussman (McGraw-Hill, 2nd edition, 1996).
Sounds to me like publishing oneliners on NPM is a trivial way to build a botnet.
> If npm was invoked with root privileges, then it will change the uid to the user account or uid specified by the user config, which defaults to nobody. Set the unsafe-perm flag to run scripts with root privileges.
The majority of code I have ever seen is awful (20 years across large + small companies) but that is why I am hired to fix awful code so I am skewed. The amount of times I have seen people implement something simple, in a convoluted error prone was is unbelievable.
I know this seems ridiculous but when you see time and time again how people fail to do the simplest things it seems like a good idea.
Why is this insane? What alternatives would be better?
isPositive = (x) => +x === x && x > 0
Keep in mind though that absolutely not all JS programmers are like that. Not everyone wants to be an aforementioned Dick-from-a-mountain.
A good micro-module removes complexity. It has one simple purpose, is tested, and you can read the code yourself in less than 30 seconds to know what's happening.
Take left-pad, for example. Super simple function, 1 minute to write, right? Yes.
But check out this PR that fixes an edge case: https://github.com/azer/left-pad/pull/1
The fact of the matter is: every line of code I write myself is a commitment: more to keep in mind, more to test, more to worry about.
If I can read left-pad's code in 30 seconds, know it's more likely to handle edge cases, and not have to write it myself, I'm happy.
The fault in this left-pad drama is not "people using micro-modules". The fault is in npm itself: all of this drama happened only because npm is mutable. We should focus on fixing that.
IMO in the Javascript world they're only there in order to minimize script size for front end work. See lodash & lodash-something1 / lodash-something2 / ..., where there's an option of using the whole module or just including 1-function long scripts, precisely to avoid the script size issue.
Is there a solution for this? I know that the Google Closure compiler can remove dead code, ergo making inclusion of large modules less costly in terms of code size. Am I missing some ES6 feature that also helps with this?
Left padding is (almost in all languages) built-in, even C can do it with printf (edited)
The problem is not having a library that offers that, but having this micro-module thing as a whole NPM module. No other language does that.
If it was inside a string helpers, that's great.
But don't make me one single module for just left-padding (or is-integer-number)
JavaScript had a very minimal standard library, it's pretty asinine of you to compare it to C or any other language with a pretty extensive standard library.
It's easy to say "I don't see why there's a need for an 11 line module to pad left on a string" when your language of choice has a robust standard library with such abilities built in.
1) JS is unique in that it is delivered over the wire, so there is a benefit in having micro-modules instead of a bigger "string helpers" module. Things like webpack are changing that now (you can require lodash, and use only lodash.padStart).
2) JS's "standard" library is so small, because it's the intersection of all of the browser implementations of JS dating as far back as you care to support. As pointed out in sibling, a proposal for padLeft is included in ECMA2016. But we'll still need Left-Pad for years after it's adopted.
Also, the Haxe language, which compiles to JS, has DCE.
Micro-modules is certainly a solution if you don't want to use pre-processing or compilers. So is copy/pasting, or manually writing your own util libs, which seems safer than relying on third parties to provide one-liners for you.
var obj = {
a: ...
b: ...
c: ...
}
var prop = webrequest();
obj[prop]();
I've heard ES6 modules solve this problem, but it seems like dynamic access to an ES6 module might still be possible, which would cause the same problems for DCE. Perhaps no one writes code that way, so it doesn't necessarily matter. But what about eval?
There are lots of tricky corner cases. It seems better to use small atoms than a monolithic package.
In advanced mode, Closure Compiler would globally rename properties and statically eliminate dead code. In your example, it would remove a, b, and c and break the dynamic invocation.
This behavior is all outlined in Closure's documentation with examples.
[1]: https://developers.google.com/closure/compiler/docs/limitati...
DCE will help you if you pull in huge-monolithic-package and only use one function from it. In that case its basically the same as if you had used the micro package.
Since the vast, vast majority of Javascript doesn't work with it, the issue certainly wasn't addressed years ago.
Yes, you're right, making use of dynamic behaviour will break DCE.
The wise course of action would be to take those very good ideas and bake them in languages designed for web/system programming, and keep improving JS for in-browser tasks.
Yes you can use left-pad module in a browser application using the npm install infrastructure.
JS isn't even remotely unique in this regard, almost every static language has had dead code removal for decades.
https://github.com/sindresorhus/ama/issues/10#issuecomment-1...
With npm and the popularity of small modules, it was obvious that I could just cheaply publish each of my utility functions as separate modules. Some of them are about a few dozen lines, but have hundreds of lines of tests and have had significant bugfixes that I am very happy that I haven't had to manually port to dozens of projects. I don't miss copy-pasting code across projects, no matter how many claim I've "forgotten how to program".
There is something about JavaScript that makes people go a little crazy both for and against it.
I've never seen so many programmers advocate copy/pasting code before...
But regardless of how many insults get thrown around, or how many people seem to think JS is useless or that it's a horrible language, its probably my favorite (and I've done professional work in non-trivial applications from C and C++, to Java and go, to python, Ruby, and PHP to BusinessBasic and even some lisp).
I'm going to keep writing stuff in JS, and I'm going to keep loving it. Regardless of how many people are telling me I'm wrong.
Why do you like it? Serious question.
This got a lot bigger than i thought, so strap in!
* The lack of "private" anything. This sounds like a bad idea, but I firmly believe it was a major reason for JS's success. The ability to "monkey patch" anything including built-in functions and other libraries means that everything is extendible. It isn't something i do very often (mucking around with internals of another module/system) but when i do it's really fun and generally solves a problem that otherwise would be unsolvable.
* The debugging. Oh the debugging! It's magnitudes better than anything i've ever used before. And i don't just mean in features (i know that other langs have feature X that JS doesn't have, or can do Y better). I can use multiple debuggers, inspect EVERYTHING, breakpoints, live inline code editing, remote-debugging (on pretty much every mobile device), world-class profiling tools with memory usage, cpu usage, JIT performance, optimizations/deoptimizations, etc... Hell Edge is even getting "time travel debugging" where i can step BACKWARDS in code, edit it in place, then play it forward again! Also, sourcemaps! I can compile python/coffeescript/typescript/javascript to javascript and then minify it and combine multiple files, but when i open the debugger i see my source files, with their full filenames, and the execution is synced statement-by-statement. And they work for CSS too! And I almost forgot about the best part. Since they can be separate files, i can include them in my production build with literally 0 overhead. So if there are any problems with the live build, i can open the debugger and have the full development-mode first-class debugging experience, on the live site, even on a user's PC if i need to. Hell i can even edit the code in-place to see if my fix works! This one is probably one of my favorite features of javascript and it's ecosystem.
* async programming. Yeah, i know other languages have it, but JS is the first time where i would consider it a "first class citizen" Everything is async, it's amazing, and it's to the point that if something isn't async, it's almost a bug. And this combined with the event system and the single-threaded-ness means writing performant code is more "straightforward" than i've experienced in other languages. Combine this with web-workers (or threads in the node ecosystem) and you get the best of both worlds.
* the mix of functional and OOP programming. Functional programming sucks for some things, OOP sucks for others. I feel like in practice JS lets me use the best of both. Yeah, it's not "pure" or "proper", yeah you can use the worst of both, but i love it. You can add the mix of immutable vs mutable in this as well. By having both, it lets me choose which i want to work with for the current problem, even switching in a single project.
* it's fast. JS is pretty fucking fast in the grand scheme of things. Yeah, it's not C, but with typed arrays and some profiling (which JS makes oh so easy!) it's possible to wipe the floor with Python, Ruby, PHP, and can even give Java and Go a run for their money. For such a dynamic language, that's impressive.
* the compilation options. From coffeescript/typescript/flow, to just compiling between js "dialects", and adding non-standard (or extremely new) features to the language is "easy". It took me a little while to get used to being that disconnected from the final output, but once i "let go" of that, i found i loved it. With babel plugins i can add extra tooling, or extra type-checking, or even completely non-standard stuff like JSX or automatic optimizations into the code that i output. Combined with some good tooling i can even change how the code executes based on the "build" i'm generating (for example, i have some babel plugins that optimize react elements to improve first-load speed, but i only run it on staging/production builds because it is pretty verbose (which gets removed when gzipped) and is difficult to debug)
* the tooling. auto-refresh, hot-module replacement, automated testing, integration testing, beautiful output in multiple formats, linting, minifying, compressing, optimizing and more task runners than you'll ever need. The fact that i can write code, save, and have that code hot-replace the code currently running in my page on my PC, tablet, phone, laptop, and VM all at the same time. There is nothing that even comes close to this. At all.
* and i guess finally, npm. The fact that there are 5+ different modules for everything i could ever want. The fact that i can choose to install a 3-line program, or write it myself, or install it first and write it myself later, or vice versa. The fact that i can choose a module optimized for speed, or one for size. The fact that i can get a pure-js bcrypt and a natively-compiled bcrypt with the exact same API and install the native and if that fails fallback to pure-js. The fact that NPM installs are so effortless that i have a project with about 100 direct dependencies (and most likely about 1000 when it's all said and done), and there isn't even a hint of a problem is wonderful (this is a bit of an edge case though, most of the packages installed here are "plugins" like babel plugins, postcss plugins, and i'm purposely avoiding bundled deps kind of for shits-n-giggles.) And no matter how many internet commenters keep telling me i'm wrong, i haven't had any issues with it.
This got a lot bigger than i had intended, but the point is that while JS might not do any one thing very well, it does many things pretty damn well. And the benefits far outweigh the downsides for me.
I'm going to bed for the night, so if you reply don't expect an instant reply, but despite the "standoffish" nature of a lot of this, I want to hear responses.
To add to the part about extensibility, many times I have jumped into a "node debug" session, in my own code or in 3rd party modules installed to node_modules. Many times I have added breakpoints or tweaked 3rd party code right in node_modules, or monkey-patched methods temporarily. This kind of thing is often nearly impossible, very time consuming, or just plain difficult to do in other languages.
"The lack of 'private' anything" - Just hearing this caused immediate revulsion and I am sorry to say that. Where I come from, the best practice is to try to keep everything closed from modification but still open enough for extension: http://www.cs.utexas.edu/users/downing/papers/OCP.pdf
"The debugging" - The features you mentioned are all available in the Java ecosystem as well. It is a very mature ecosystem with great IDEs, debuggers, performance testers, etc. The step-back and edit feature of debugging has been around for awhile now. Heck, you can even write your own debugger fairly easily due to good support of other tools in the ecosystem.
"async programming" - Not sure what you mean by "first-class citizen", but asynchronous programming can also be done with Java as well. Callbacks and futures are used widely (at least where I work). But even better: Java is multi-threaded. What happens to the Node.js server if a thread is getting bogged down?
"the mix of functional and OOP" - I admit I have no experience with functional programming so I can't say anything about mixing the two paradigms together. But I have seen OOP with Javascript and frankly, it is confusing and unwieldy. I don't even think the concept of class as an object blueprint exists. How do you even create a class that inherits the properties of another Javascript class? It is one of the basic ideas of OOP but I don't think Javascript supports it. From my brief time with it, it really looks like you only have simple objects with properties and methods which can be set up off of a prototype but that's it.
"it's fast" - Not sure where you got this idea. Looking at various benchmarks on the internet, they show that Javascript is significantly slower (sometimes by an order of magnitude) compared to Java, Go, and C++. I did notice that it looks to be faster than Python and Ruby. https://www.techempower.com/benchmarks/#section=data-r12 https://benchmarksgame.alioth.debian.org/u64q/javascript.htm...
"the compilation options" - I'm assuming you're are talking about transpilers and yes, I've been noticing more transpilers that target Javascript. I honestly don't know why one would want to do that though. It just seems an unnecessary layer. Why not just directly write Javascript code? Is the Javascript syntax so bad that you want to code in pseudo-Ruby (Coffeescript)? :)
"the tooling" - Hot swapping, test frameworks, linting, optimizing, ...these are also available in the Java ecosystem and have been for quite some time now. Notice I didn't mention auto-refresh, minifying, and compressing since I am not sure what exactly those are and I don't think they apply to compiled languages.
"npm" - The available libraries in the Java ecosystem is vast and a great number of them have been developed and iterated upon by some of the best engineers and computer scientists in the past ~20 years. And the Java libraries do not seem to have the problems that npm is suffering at the moment :P
per the debugging, i might have to take another look at this. I hadn't realized that it was that nice!
per async, yeah java can do async programming, but in js you MUST do async programming. Because of the single-threaded nature, if you aren't async you are blocking which ruins the performance instantly. This means that every library, function, and module is built for async from the start. "What happens to the Node.js server if a thread is getting bogged down?", it runs slowly or in some cases not at all. Yeah, that sucks, but this constraint forced better and more widely available async code. Plus if you really need multiple threads you can have them (webworkers on the web, threads on node), but you need to control them absolutely. It's more work, but it's the same outcome. I'd prefer this to be different, but trying to bring multi-threaded execution to javascript is like trying to stop a tornado with your bare hands...
per functional/oop, JS's OOP is lacking (or was, recently with ES6 it's gotten MUCH better). Now you can inherit from another class, now you can use a class as an object blueprint. There's still work to be done here, but it's getting better. That being said, there are ways to solve those same problems, but they are functional. And even though we are getting inheritance, it's largely an anti-pattern (at least to me). Composition is almost always prefered. Going back to my monkey-patched library from above, i was able to modify the object at runtime to add more functions and to "redirect" others. In something like Java i'd need to wrap that class in another class and add what i want there. It's largely the same result when used alone, but when you combine that, with a polyfill that's fixing something in all objects, and with a plugin for that library (where the lib wasn't meant to have plugins in the first place), in Java land you quickly end up with a big mess, while in JS land you can keep hanging more and more crap on an object if you want at runtime. And because the language was "meant" to be used this way, everyone expects and guards against it.
per speed. It's not as fast as java/go 90% of the time, but there are times where it can be. Yeah, they are the minority, but i was more just trying to dispel the myth that JS is dog slow. It's impressively fast these days. My favorite set of benchmarks to show off is the asm.js suite from "arewefastyet"[1]. It's more for comparing JS engines against each other, but there is a "Native C++" compiled version of the same code and they are comparing their JS against that.
The compilation options. It seems like an unnecessary layer, but in practice it's not as bad as many make it out to be. You still need to know JS to do it, so you are doubling the amount of languages you need to know to work on the project, but it does allow for some cool stuff. Typescript (microsoft's strong-er typed javascript/C# hybrid compile-to-js language) actually pushed a bunch of features into the newest version of javascript. Coffeescript did as well. These compile-to-js langs are partly a symptom of a problem, and by their nature they let people "solve" the problem now and when it gets fixed they can migrate back to "pure" js if they want. Also, it's this "transpiling" that lets things like React's JSX to exist. Adding entirely new, unrelated to javascript itself, parts to the language. JSX allows you to embed HTML directly into JS and it's basically a wrapper around the "document.createElement" function (in reality it's MUCH more, but that's the gist of it). It's really strange if you haven't used it before, but it's extremely powerful. And it could be done in other languages (and it is, look at go's generate command), but it's already here in js, and i love it!
the tooling, java is probably the only other one in my "list of languages" that is on the same level as JS in terms of tooling. The problem i have with Java's tooling is they tend to be built into IDEs instead of standalone tools. So that means i'm married to my IDE if i want those features. In JS land for the most part they are standalone and can be used with many different editors. This is a pain-point at my work currently as we are switching from a Business basic stack that marries you to the IDE and everyone wants different things in their next editor. It's a small problem though in the grand scheme of things, but it's a bit of a pet-peeve of mine.
and npm. Maven is great, but it just doesn't have the same number of options that something like NPM has. I know this isn't the languages fault (the best package manager doesn't mean shit if there are no packages), but it's a pain point. Many people seem to think of having "too many options" as a problem, but I think i'm spoiled by it now. If i want a lib to handle authentication in my app, i have 5 or more major choices. They all have their upsides and downsides, they are all made for different use cases. In something like the java world i just haven't seen that amount of choice. The only other one that comes close is surprisingly go. I really think the number of packages stems from ease of publishing, and npm (and go) have that down pat. I also think that this comes down to the languages being more oriented towards different things.
I appreciate the comment, and i'm in no way trying to say that JS is the only one that has these things (not that it sounded like you were implying it), but when combined, it makes for a nice experience.
[1]https://arewefastyet.com/#machine=28&view=breakdown&suite=as...
> I can use multiple debuggers
Multiple browsers means there are multiple competing sets of debugging tools, each are good at some things and worse at others. For example, Firefox was among the first to be able to properly debug promises, while chrome still let it swallow unhandled errors.
After writing that, i think the reason i love working with JS is because of the choice. A lot of that choice isn't necessarily because of the language (you could easily have that debugging experience in other languages), but it's currently in javascript.
Regarding a lack of private anything, it's possible to monkey-patch any Lisp package or class however one wants. And of course, one can get true privacy in JavaScript if one wants, by using closures — the same trick applies in Common Lisp.
Lisp debugging is great: one can set up all sorts of condition handlers and restarts, and invoke them from the debugger.
Lisp is a great blend of imperative, functional & object-oriented programming styles, enabling you to use the right tool for the problem at hand.
Lisp is incredibly fast, faster than C & C++ in a few cases and almost always Fast Enough™. It's dynamic, but inner loops can be made very static for performance. There's even a standard way to trade off safety and performance, if that's what's important in a particular case.
I don't know for certain, but I believe that Lisp was the language that invented hot-patching (well, I suppose one could always have done it from assembler …). It was even used to debug a problem on a NASA probe[0]: 'Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem.' As with JavaScript, the Lisp debugger is part of the standard and is always available. This can be profoundly useful.
And Quicklisp is a nice, modern way to pull down numerous libraries.
Lisp does of course support async programming since function are first-class, although I'm not personally as much of a fan as you are. Generally, I think that callback hell should generally be avoided.
I'm not aware of a lot of compile-to-Lisp projects, but given that the language is great at treating its code as data, it's an excellent target.
It certainly doesn't have the huge ecosystem that JavaScript does, but that improves with every developer who starts a project.
I really, really wish more folks would take a look at it. The more I use it, the more I realise that a 22-year-old standard has well-thought-out solutions to problems that people still face in other language environments today.
The only thing i have to add is that while callback hell sucks, there have been some pretty recent (in the grand scheme of things) additions to the async programming field. Async/await is beautiful, and it has made me fall in love with async programming all over again.
I see one common thread between all those languages you list: none of them has a decent type system.
If you ever get the chance I'd strongly recommend trying a small project in a strongly-typed functional language - PureScript if you're targeting the browser, otherwise Haskell or Scala, or maybe F# or OCaml. (Scala and OCaml also have compile-to-JS options). If you find you don't like that style then fair enough, but it's well worth trying one language in that space and getting a sense of the techniques that it enables - it will make you a better programmer even if you end up going back to JavaScript or another language from your list.
I might shoot for Scala next time. We don't use Java anywhere at my current job, but I might play around with it in a personal project for a while.
I really like the functional style, and I can see how strong typing works REALLY well with it, but I've already found that it's pretty hard to bring other devs up to speed on it. And that really limits where I use it.
That way, different projects can have different versions of everything, and system environment is only important to the synchronization step - trivial to fix up if needed, trivial to copy between machines.
So, when I read "It doesn't matter if the module is one line or hundreds." I call bullshit. There is overhead, it's usually just fairly small (at may event begin to rival the gains from using a module at that level), but that small amount adds up. Once you've had to deal with a dependency graph that's 10 levels deep and contains hundreds or thousands of modules, that small extra complexity imposed by using a module is no longer in total, and comes at a real cost, as we've just seen.
Other module ecosystems have gone through some of the same problems. There was a movement in Perl/CPAN a few years back to supply smaller, more tightly focused modules a while back, to combat the sprawling dependencies that were popping up. The module names were generally suffixed with "Tiny"[1] and the goals where multiple:
- Where possible, clean up APIs where consensus had generally been built over what the most convenient usage idioms were.
- Try to eliminate or reduce non-core dependencies where possible.
- Try to keep the modules themselves and their scope fairly small.
- Remove features in comparison to the "everything included" competitor modules.
This has yielded quite a few very useful and strong modules that are commonly includes in any project. They aren't always tiny, but they attack their problem space efficiently and concisely. Even so, I'm not sure there's ever a module that's a single line of code (or less than 10, given the required statements to namespace, etc), as the point is to serve a problem, not an action.
1: https://metacpan.org/search?size=20&q=%3A%3Atiny&search_type...
That's true. However:
Every dependency you add to your project is also a commitment.
When you add a dependency, you're committing to deal with the fallout if the library you're pulling in gets stale, or gets taken over by an incompetent dev, or conflicts with something else you're using, or just plain disappears. If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself. That's why so many people are shaking our heads at a 17-line dependency. It's way more risk than it's worth. If you need a better stdlib for your language (some of us write PHP and feel your pain) then find one library that fills in the gaps and use that.
Actually that's not correct when using node/npm (or anything else in the ecosystem like browserify). That is one of the impressive things about this platform: any number of different versions of the same module can be required by the same app. It would be nuts to do that in your own code, but as long as the craziness is in your dependencies it really doesn't matter.
Only one buggy implementation per project. Compare this to including the same library in dozen different versions, because dependencies have their own dependencies. And you can neither track the versions nor update them.
Using multiple versions of the same library is code smell. It's a stability issue, a security issue, a complexity-breeder, and an absolute nightmare for packagers.
And npm allows it to happen silently.
Actually though you're just talking about some bugs in X, or possibly some design flaws in Y. Passing [EDIT, because bare objects are fine:] class instances around like that is the real code smell. So much coupling, so little cohesion. We call them "modules" because we like modularity.
There is no clean way of packaging a typical node application.
Passing class instances around like that is the real code smell.
Often, yes, but not always. Allowing fine-grained control over performance is one good reason that a library might expose class instances from one of its dependencies.
A distro that still packages python 2 (i.e. all of them) has a particular "velocity", and therefore it has no business packaging Node or anything written in it. Maybe a distro could package TJ's "n" tool (helpfully, that's written in bash rather than node), which would actually be handy for distro users who also use Node, but that's it.
For example, you can install Wordpress on Arch with `pacman -S wordpress' and you'll have a managed wordpress installation in /usr/share/webapps/wordpress. Then you just edit some wordpress config files, set up your http server to serve php from that directory, and you have a wordpress blog.
It would be nice to be able to do the same with Ghost.
I'm sure if I look hard enough there are some silly idiosyncratic steps one might take to install this module. Suffice it to say that it's not installing the "npm way", so it's misguided to blame npm for packaging difficulties.
More generally, I can certainly understand distro packagers' refusal to recreate towering pyramids of node dependencies in their own package system. Some lines have to be drawn somewhere, and "major" modules must bundle many of their dependencies when packaged for distros. If module maintainers don't do this, and distro packagers can't, then the modules can't be packaged.
If there is a security fix, you should bump your dependency by hand, the other problems that you pointed out do not exist in Node (and it's about time they disappear in Java)
If its just a few lines of code, just copy the thing into your code base? throw a comment in saying "came from xxxx" so anyone reading your code knows that it might look like a overly generic function because it is.
This is a problem with NPM, not with dependencies. With different package management systems with stable builds and lockfiles, then you pin to a specific version and there is no way upstream can cause problems. A lockfile is a pure win over vendoring.
The only way to prevent this is to have your own local server for third party package repository.
For example, I was a maintainer for an admin UI component of the last version of Drupal core, and we decided to pull in a query string handling library written by Ben Alman. It was a good decision, and it hasn't caused any problems. But it still meant we were trusting him to maintain part of our codebase. It was also an implicit commitment to every user of Drupal that if Ben quit maintaining that library, we would step in and fix any problems that came up. You don't get rid of that commitment with a lockfile.
A dependency introduces some constant amount of risk (d) that does not vary with the size of the dependency. Every line of code you write yourself also introduces a much smaller constant amount of risk (y).
If you introduce a separate dependency for every line of code in a 1000-line project, your risk is 1000d.
If you can pull in someone else's code for the whole thing and don't need to write any code yourself, your risk is d.
If 200 lines of your code can be replaced with an external library, your risk is d + 800y.
I think the real disagreement here is over the value of d. My experience leads me to put the value of d pretty high relative to y, so to me 1000d is the worst possible case. If someone sees d as equal to y, then they'd see dependencies as no problem whatsoever.
(Obviously in reality the risk of a dependency is not really constant - it's probably more like d + 0.1y or d + 0.01y or whatever, since a 10-line dependency is less risky than a 1000-line dependency. Hopefully my point still stands.)
Honestly, and this is maybe just me being biased against JavaScript, then this is what happens when you pick a language fully knowing it's limited in all sort of silly ways and attempt to use it as a general purpose language. It's not that you can't, but if you need say typing that can tell you if something is an array, then maybe picking JavaScript to begin with wasn't the brightest idea.
There's a ton of libraries and hacks out there, all attempting to fix the fact that JavaScript isn't really good general purpose language. ES6 is fixing a lot of these thing, but it's a little late in the game.
Why do you need an external dependency on something so small?
The whole "amaga, it's a whole package for just ten lines of code" is just elitism. Given the number of downloads on things like left-pad, it's clearly useful code.
I want to plug ied (https://github.com/alexanderGugel/ied) here, which both installs about 5x faster than npm@2, yet deduplicates as well as npm@3.
Looking at that left-pad module though - no comments, abbreviated variable names, no documentation except a readme listing the minimally intended usage examples. This is not good enough, in my opinion, to upload to a public repository with the objective that other people will use it. It is indistinguishable from something one could throw up in a couple of minutes; I certainly have no reason to believe that the future evolution of this code will conform to any "expectation" or honour any "commitment" that I might have hopefully ascribed to it.
[EDIT: I've just noticed that there are a handful of tests as well. I wouldn't exactly call it "well tested", as said elsewhere in this thread, but it's still more than I gave it credit for. Hopefully my general point still stands.]
The benefits of reusing other people's code, to a code reuser, are supposed to be something like:
(a) It'll increase the quality of my program to reuse this code - the writer already hardened and polished this function to a greater extent than I would be bothered to do myself if I tried right now
(b) It'll save me time to reuse this code - with the support of appropriate documentation, I shouldn't need to read the code myself, yet still be able to use it correctly and safely.
Neither of those things are true for this module. It's not that the module is small, it's that it is bad.
(True that npm's mutability is a problem too - this is just a side-track.)
And just because some random guy didn't get something as trivial as this right the first time, doesn't mean nobody else can. Also the de facto standard library lodash already has padding utilities, made by people who have a proven track record.
Combining this with isomorphic code, cross-browser development and micro-services.
I even wrote a blog post couple of days ago about it [1].
1 : http://www.drinchev.com/blog/increase-your-dependencies/
NPM/JS has subsumed the class of programmer who would have previously felt at home inside PHPs battery-included ecosystem. Before that, a similar set of devs would have felt at home with Visual Basic. Seriously, go visit the comments section on archived copies of the PHP documentation. You'll find code of a similar nature. If PHP had had a module system 10+ years ago you would have seen this phenomenon then. Instead it was copy and paste.
This isn't elitism, it's just the way it is. The cost of a low barrier to entry in to a software ecosystem is taking in those who don't yet have software engineering experience.
Nobody should be surprised that NPM, which I believe has more packages than any other platform, is 90% garbage. There are only so many problems to solve and so few who can solve them well, in any language. Put 100 programmers in a room, each with 10 years experience, and you'll be lucky to find 1 who has written a good library. Writing libraries is really hard.
Compare NPM to Composer/Packagist and you get a better comparision. I've personally seen only very few micro packages on Packagist, thankfully this never seemed to gain traction in the PHP world.
Add into that the fact that:
1) Javascript has a huge number of developers, and is often an entry-level language
2) The developers on this thread (I like to think of HN as at least slightly above average) are divided whether having small packages / large dependencies trees is a good or bad thing
3) Dependency management is something that matters mostly to long term (professional / enterprise / etc) applications, which is a subset of programming, and I wonder if not a minority subset of node.js projects in general.
4) If I'm writing a throwaway app or proof of concept, and therefore don't care about dependency maintenance, using as many dependencies as possible is a major time savor,
and of course you get this situation, and it seems to make perfect sense.
Personally, I wish there was an NPM Stable, where packages underwent much more scrutiny and security in order to get it, but nonetheless, nothing I've read so far about npm really scares me given the the above context. If you are a dev creating an unmanageable dependency tree for your enterprise app, you're a shitty dev. That doesn't necessarily mean that NPM is wrong for being so open in allowing others to publish their packages, or that smaller / more worthless packages shouldn't be allowed to publish.
That said, I would really like to hear a response to this post, as I have limited experience with different package management systems.
Can you expand on how to identify the class of programmers you're referring to? Are they the type that copy / paste code directly from StackOverflow? They lack a classical computer science education? They haven't worked on a large, enterprise-grade project?
Others see programming more as an art. They take care to make the code not only efficient but also elegant. They'll read up on new and interesting algorithms and incorporate them in novel ways. They might often be behind deadlines, but when they are, they create things like GNU Hurd that inspire a lot of interest and lead to interesting results, maybe even teach people a few things. Their code is interesting to read. They tend to write the libraries that the first group uses.
Both groups contribute a lot, but it's not easy to get them to understand that about each other.
1/ forces package authors to write stable libraries
2/ forces dependencies to narrow the versions of their dependencies
3/ prevents name squatting to some extent. You cannot have a package named "forms" and then sell the name for real money, like seen on NPM. your package needs to be "namespace"/"name". NPM made a huge mistake with its gems like global namespace and it explains half the problems it is having today.
When you can't even reason about the truthiness of a variable because the language coerces everything, of course things end up screwy.
a) No standard lib in JS
b) JS is delivered over the internet to web pages in a time sensitive manner ... so we don't want to bundle huge "do everything" libs. Sometimes its convenient to just grab a tiny module that does one thing well. There isn't the same restriction on any other platform
c) Npm makes it really easy to publish/consume modules
d) And because of c) the community is going "all in" with the approach. It's a sort of experiment. I think that's cool ... if the benefits can be reaped, while the pitfalls understood and avoided then JS development will be in an interesting and unique place. Problems like today can help because they highlight the issues, and the community can optimise to avoid them.
Everyone likes to bash the JS community around, we know that. And this sort of snafu gives a good opportunity. But there many JS developers working happily every day with their lots of tiny modules and being hugely productive. These are diverse people from varied technical backgrounds getting stuff done. We're investigating an approach and seeing how far we can take it.
We don't use tiny modules because we're lazy or can't program, we use them because we're interested in a grand experiment of distributing coding effort across the community.
I can't necessarily defend some of the micro modules being cited as ridiculous in this thread, but you can't judge an entire approach by the most extreme examples.
However, I agree it is ridiculous to have a dedicated module for that one function. For most nontrivial projects I just include lodash, which contains tons and tons of handy utility functions that save time and provide efficient, fast implementations of solutions for common tasks.
Lodash includes `padStart` by the way (https://lodash.com/docs#padStart).
Nothing was done in a way to pull the rug out from underneath folks.
I think that was largely the OP's point tbh. Using something like lodash [a utility library] is fine while using a module [for a single function] is not.
It might have gotten lost in the ranting from on high but I don't think the author truly meant more than that.
(And, y'know, maybe it's because I'm not a JS programmer, but the notion of looking for a module to implement a string padding function would never have even occurred to me.)
a Git submodule like approach would be much better
My point is OSS is a collaborative effort by often times anonymous contributors. Therefore there will always be a risk of bugs or back doors - regardless of the distribution mechanism.
In my opinion any criticism against micro packages is equally valid against large packages. There's no guarantee that a pull request would receive any more scrutiny than a external dependency. I mean look at heart bleed. Surely this doesn't mean that OSS is broken, but rather stricter security protocols should be in place. My 2 cents.
Because it's so trivial? I can't wrap my head around why this is an argument in the first place. It makes no sense to bring in a module from a third party adding yet another dependency and potential point of failure when reimplementing it yourself literally takes as long as it takes to find the module, add it to package.json and run npm install.
People should be trying to limit dependencies where possible. Reproducible builds are really important if it costs you almost no time you should have it in your code base IMO.
People taking the DRY principle to the most extreme degree always makes for the worst code to debug and maintain.
Even if it does take the same amount of time (which it shouldn't), a 1-line call to a standard module imposes less of a future maintenance burden than 14 lines of custom code.
> People should be trying to limit dependencies where possible. Reproducible builds are really important if it costs you almost no time you should have it in your code base IMO.
That's a non sequitur. Reproducible builds are important, but unless you write code with 0 external dependencies you already have a system in place for handling library dependencies in a reproducible way. So why not use it?
> People taking the DRY principle to the most extreme degree always makes for the worst code to debug and maintain.
This is the opposite of my experience.
In my experience with using npm since it's release, module authors will spit out modules very quickly then, after some period of time, abandon them without passing them onto other people. At which point I have to assume all future maintenance anyway. This has happened to me so many times, in fact, that I try to make even picking my dependencies based on the author's interests. For example if it's owned by a company or organization that still uses the module then it's usually one of the safest to pick.
Regardless I don't think I'd ever call very elementary code a "maintenance burden". Ever.
> That's a non sequitur. Reproducible builds are important, but unless you write code with 0 external dependencies you already have a system in place for handling library dependencies in a reproducible way. So why not use it?
Completely disagree here. As we saw with this "npm gate", even if you're using a shrinkfile, npm doesn't completely provide handling dependencies in a reproducible way. Not always. Maybe most of the time though our build server certainly has logs where npm was unreachable, having issues, etc on a very regular basis.
The point being: where it's possible to mitigate and remove dependencies I think you'd be crazy not to. Every dependency you can lose is another potential build issue or attack surface you're removing from your project.
> This is the opposite of my experience.
That's fine. In my experience people will take DRY so far that even meta data and comments will be abstracted so you can't even understand a piece of code without opening multiple files. I think it's perfectly reasonable to repeat yourself at times but those cases where you have to open up 5 files just to understand what a REST endpoint accepts as input is crazy.
I think DRY in general is fine as long as it's not used as an absolute "we have to do it this way because DRY". :)
Every line is another line that maintainers have to read and understand.
> The point being: where it's possible to mitigate and remove dependencies I think you'd be crazy not to. Every dependency you can lose is another potential build issue or attack surface you're removing from your project.
Disagree. If there are issues with npm shrinking not working then you absolutely need to resolve them - but resolving them is an O(1) problem no matter how many dependencies you have. Just like if you've already written a good general-purpose sorting function, there's no point writing a separate integer sort routine, even if the implementation could be simpler than the general-purpose one. You already depend on your packaging/dependency tools, so you might as well use them all the time.
Seriously? This is what you're going with? We're talking about very simple, elementary programming. To be worried about maintaining code that you learn how to do in the first few classes of any programming 101 class is absolute insanity.
> Disagree. If there are issues with npm shrinking not working then you absolutely need to resolve them
Can't resolve them if the module disappears or is replaced with a malicious module. Nor if you or npm are having connectivity issues (which, on the npm side, happens very frequently).
> Just like if you've already written a good general-purpose sorting function, there's no point writing a separate integer sort routine, even if the implementation could be simpler than the general-purpose one.
Not sure what you're getting at. If you need a sorting function you can probably use whatever is built into the language unless you need to sort across a distributed data set in which case write something or find a dependency to use.
> You already depend on your packaging/dependency tools, so you might as well use them all the time.
Absolutely, unequivocally, no. What you're saying is you should install and use dependencies, from third parties which you do not know anything about, for every single, possible thing just so you can use the tools "all the time". That's so irresponsible and backwards.
Use the tools for the job they were meant to be used for. Need a dependency because someone can do it better / faster / cheaper then you? Then grab it by all means. But don't use it for every tiny function and for loop just because you want someone else to maintain it.
Every line is a maintenance burden - just reading and understanding the code is what takes most of the time. Lines of code (and notably not any measure of "complexity" of those lines that's been tried) is the one thing that correlates with bug rates.
> Can't resolve them if the module disappears or is replaced with a malicious module. Nor if you or npm are having connectivity issues (which, on the npm side, happens very frequently).
So figure out a process and resolve that, once and for all. There's no point just cutting 10% (say) of your dependencies and hoping that you won't encounter the problem on your more important dependencies.
> Use the tools for the job they were meant to be used for. Need a dependency because someone can do it better / faster / cheaper then you?
That's exactly why people were depending on this library.
I'm sorry but that is just a horrible way to look at programming.
You shouldn't NEED to go out and look for an already done solution if it's elementary and takes minutes, if that, to write. Ever.
This is just sloppy.
> So figure out a process and resolve that, once and for all.
Tell yourself that; that was simply a counter argument to your false claim. Regardless it's fixable.
> There's no point just cutting 10% (say) of your dependencies and hoping that you won't encounter the problem on your more important dependencies.
This doesn't even make sense. What are you trying to convey here? The more dependencies you can cut out the more reproducible your builds will be. Period. Which is important when you're dealing with code that gets rapidly deployed to many production boxes.
> That's exactly why people were depending on this library.
If using left pad gave them time back because the original author could do it better, faster and cheaper...I'm not sure programming is the right type of work for these people.
What's next, are you going to outsource all your for and while loops to a module? You know, so you have less things to "maintain"?
If writing it would take minutes and adding the dependency would take seconds, add the dependency. And how long it takes to look for is beside the point - code is read more than it's written, so how long it takes to read is much more important.
> This doesn't even make sense. What are you trying to convey here? The more dependencies you can cut out the more reproducible your builds will be. Period. Which is important when you're dealing with code that gets rapidly deployed to many production boxes.
No, look, if you have some kind of problem where dependencies maker your builds unreproducible or break your deployments, you need to fix that problem. If you have that problem when you have 100 dependencies, you're still going to have that problem when you have 90 dependencies. Unless you're going to cut every dependency, cutting dependencies is not the way to fix that problem.
> What's next, are you going to outsource all your for and while loops to a module? You know, so you have less things to "maintain"?
for and while probably should be ordinary functions (smalltalk style) and probably should be in a library somewhere rather than having everyone reimplement them, yes. Almost all languages have a for or while in their standard library so I don't know what you're really saying?
I'll tell you why.
The least important ones is that downloading such trivial module wastes bandwidth and resources in general (now multiply this by several hundred times, because of dependency fractal JS sloshes in). I would also spend much more time searching for such module than I would implementing the damn function.
More important is that you give up the control over any and every bug you could introduce in such trivial function or module. You don't make it less probable to have those bugs (because battle-tested package! except, not so much in JavaScript, or Ruby, for that matter), you just make it much harder to fix them.
And then, dependencies have their own cost later. You actually need a longer project, not a throw-away one, to see this cost. It manifests in much slower bug fixing (make a fix, find the author or maintainer, send him/her an e-mail with the fix, wait for upstream release, vs. make a fix and commit it), it manifests when upstream unexpectedly introduces a bug (especially between you making a change and you running `npm install' on production installation), it manifests when upstream does anything weird to the module, and it manifests in many, many other subtle and annoying ways.
> It manifests in much slower bug fixing
I don't buy this at all, because I've done it myself many times. If you're waiting on a PR from the original repo owner to fix a Production bug, you're doing it wrong. It's trivial to copy the dependency out of node_modules and into your src, and then fix the bug yourself. Then when the owner accepts your PR, swap it back in. I don't understand the problem here.
Just let it crash if the user is trying to leftpad a boolean, for crying out loud.
You're working the problem around instead of having it solved. You're moving a library in your repository back and forth, while the library should never land there in the first place (or stay there until it stops being used).
But even if you don't agree with this strategy, it's still much more work than to just commit the fix and be done with it. And you still don't control who introduces bugs to your code with modules upgrades, having much bigger surface to random external programmers than you would if you only used things large enough to pay for themselves.
Battle-tested still applies - if you have that many people using a line of code they're more likely to find any bugs. (Formal proof is better than any amount of testing, but no mainstream language requires formal proof on libraries yet)
> And then, dependencies have their own cost later. You actually need a longer project, not a throw-away one, to see this cost. It manifests in much slower bug fixing (make a fix, find the author or maintainer, send him/her an e-mail with the fix, wait for upstream release, vs. make a fix and commit it), it manifests when upstream unexpectedly introduces a bug (especially between you making a change and you running `npm install' on production installation), it manifests when upstream does anything weird to the module, and it manifests in many, many other subtle and annoying ways.
Large monolithic dependencies have this kind of problem - "we upgraded rails to fix our string padding bug and now database transactions are broken". But atomised dependencies like this avoid that kind of problem, since you can update (or not) each one independently. Regarding fixing upstream bugs, you need a good process around this in any case (unless you're writing with no dependencies at all).
- if the programmer uses other functions included in Lodash his code will have a single larger point of failure. For example, if Lodash is unpublished (intentionally as in this case, or unintentionally) then the programmer will have a lot more work to redo.
- Lodash introduces a lot of code, while the programmer only needs one of its functions to pad a string.
Using a library like lodash makes a lot more sense once you use a module bundler that allows tree shaking (like Rollup or Webpack 2.0) along with the ES6 module syntax. Heck, even if you're just using babel with Browserify or Webpack 1.x, you can use babel-plugin-lodash [0] so it'll update your imports and you only pull in what you need.
That said, I work with Java, Clojure and Python mostly so I may be more used to having a huge standard library to lean on than is typical.
Edit: But there would sure be a period of chaos.
If you depend on 2,000 tiny individual modules, each from different authors, you depend on 2,000 tiny individual modules. You have 2,000 different points of failure. Any one of those authors going rogue will break your build or compromise your system, and every one of those tiny modules has a lot less attention and care paid to it than a larger library like Lodash.
_.padleft
So in less than a year "padleft" came and went away because all strings don't start on the left and someone decided that "left" means "start" except that the reason that it doesn't is the reason that it was changed. Even worse, the 4.0 documentation does not document that _.padstart renamed _.padleft. It's hard to grok what cannot be grepped.
Why blame someone for depending on padleft in a world where libraries swap out abstractions in less than a year? Breaking changes are bad for other people. Semantic versioning doesn't change that.
Dropping the this param with no period of deprecation? Pretty breaking change to make with no warning. Renaming first/rest to head/tail? Was it really worth it? Particularly when they go the opposite direction of replacing traditional functional names with more explanatory names by removing the foldr alias for reduceRight.
All this in a language that doesn't make it easy to do automated refactoring means that you basically break everything upgrading to Lodash 4.0 so you can't refactor parts to the newer style piece by piece.
For instance I added a utility to my own library (msngr.js) so I could make HTTP calls that work in node and the browser because even the fetch API isn't universal for some insane reason.
We have a large internal C++ app at my work of that vintage (~1992) it uses its own proprietary super library (called tools.h++) which is just different enough from how the C++ standard evolved that its not a simple task to migrate our codebase. So now every time we change hardware platforms (has happened a few times in last 30 years) we have to source a new version of this tools++ library as well.
I find it amusing Javascript hasn't learnt from this.
I recently had to rebuild a large RoR app from circa 2011 and it took me longer to solve dependencies issues than to familiarise myself with the code base.
Excessive dependencies are a huge anti-pattern and, in our respective developers communities, we should try to circulate the idea that, while it's silly to reinvent the wheel, it's even worse to add unnecessary dependencies.
This is your fault for expecting free resources to remain free forever. If you care about build reproduction, dedicate resources to maintain a mirror for your dependencies. These are trivial to setup for any module system worth mentioning (and trivial to write if your module system is so new or esoteric that one wasn't already written for you). If you don't want to do this, you have no place to complain when your free resource disappears in the future.
The issue isn't that the company's software has a dependency. The issue is that the company is taking for granted the generosity of others. If they did not get a reproducible build before, they should attempt to get one as soon as they are aware of the problem. If the package is no longer available, they must now accept the punishment in terms of lost staff time or dollars to work around the lack of the dependency.
1- Maintaining a mirror of dependencies can be a non-trivial overhead. In this app that I was working on, the previous devs had forked some gems on github, and then added that specific github repo to the requirements. But they did not do it for every dependency, probably they did not have time/resources to do that.
2- As a corollary to the above, sometimes the problem is not the package itself but compatibility among packages. E.g. package A requires version <=2.5 of package B, but package C requires version >= 2.8 of package B. Now I hear you asking "then how did it compile in the frist place?" probably the requirement was for package A v.2.9 and package C latest version, so while A was frozen, C got updated. This kind of problems is not solved by forking on Github, unless you mantain a different fork of each library for each of your project, but that's even more problematic than maintaining dependencies themselves.
P.S. At least for once, it wasn't "my fault", I didn't build that app LOL ;-)
You've precisely identified the trade-off. You basically have three options. You can
1. Maintain a local repo of your dependencies (high effort)
2. No dependencies, include everything as 'first-class' code (lower upfront effort, but v. messy)
3. Rely on third-party repos (easiest, riskiest)
Automatically upgrading versions (i.e. not pinning versions) in a production build is an anti-pattern.
These sound like problems incurred due to a previous lack of software engineering rigor. As an industry, when we encounter challenges like this we should be learning how to solve them without reinventing the wheel. Pinning versions and maintaining mirrors of dependencies (whether that's an http caching proxy for npm/pypi/maven/etc or keeping a snapshot of all dependencies in a directory or a filesystem somewhere) is something that any company requiring stability needs to take seriously.
In this particular case, there were just 3-4 requirements (out of more than 100) that were pointing to a git repo, and only one of them also specified a particular commit. The other "git-requirements" were just cloning the latest commit from the respective repo.
> Automatically upgrading versions (i.e. not pinning versions) in a production build is an anti-pattern.
We did not have access to a production version, only to a git repo, that's the very reason why we had to rebuild in the first place. I can imagine all versions were locked when the system went into production years ago.
Or, you can implement the functionality yourself (or copy/paste if the license allows) and avoid the hassle.
In the Ruby ecosystem, library authors didn't really start caring about semantic versioning and backwards compatability until a few years ago. Even finding a changelog circa 2011 was a godsend.
I think this was mainly caused by the language itself not caring about those either. 10 years ago upgrading between patch releases of Ruby (MRI) was likely to break something.
At least this is one thing JavaScript seems to do better.
The same goes for almost any legacy app.
I've been working on a .net web app, that's been around since 2008. It's been continually evolved, so it's running on the latest MVC framework, uses microservices etc
As result it's build up a huge amount of automated tests. The business logic has been built up from experience and is well tested even for odd cases.
You throw away a lot of stuff for a rebuild.
All kidding aside, if the concern is just for continued security patches, there's always Rails LTS [0]. You have to pay to play, but it's cheaper than a security breach or weeks of development time. But if you're dealing with significant performance issues or looking at major feature changes/additions, it might be more effective in the long-term to consider an upgrade. You just need to be aware of just how large a project that can turn out to be.
Let's be honest though, in the current trendy javascript ecosystem these people will already be two or three jobs away before the consequences of their decisions become obvious. Most of the stuff built with this is basically disposable.
I think we got to this state because everyone was optimizing js code for load time-- include only what you need, use closure compiler when it matters, etc. For front end development, this makes perfect sense.
Somewhere along the line, front end developers forgot about closure compiler, decided lodash was too big, and decided to do manual tree shaking by breaking code into modules. The close-contact between nodejs and front end javascript resulted in this silly idea transiting out of front-end land and into back-end land.
Long time developers easily recognize the stupidity of this, but since they don't typically work in nodejs projects they weren't around to prevent it from happening.
New developers: listen to your elders. Don't get all defensive about how this promised land of function-as-a-module is hyper-efficient and the be-all end-all of programming efficiency. It's not. Often times, you already know you're handing a string, you don't need to vary the character that you're using for padding and you know how many characters to pad. Write a for loop; it's easy.
Note that this is exactly the sort of question I ask in coding interviews: I expect a candidate to demonstrate their ability to solve a simple problems in a simple manner; I'm not going to ask for a binary search. Separately, I'll ask a candidate to break down a bigger problem into smaller problems. In my experience, a good programmer is someone who finds simple solutions to complex problems.
Note: rails is similarly pushing back against developers that have too many dependencies:
https://www.mikeperham.com/2016/02/09/kill-your-dependencies...
Small modules are not evidence of a problem, and they certainly aren't evidence of an inability to implement these things on the part of the people depending on them. Why would I implement left-pad myself when there is already a well-tested implementation that I can install? Building up an ecosystem of tiny abstractions, bit by bit, iteratively and evolutionarily, is how we get robust, well-designed complex systems. We don't get there by everyone reinventing the left-pad function to sate some misplaced appetite for self-reliance.
The author seems to make some arbitrary distinction between things that are 'large enough' to be packaged and 'pure functions' which are 'too small' to be their own modules, and I just couldn't disagree more. Tiny, pure functions are ideal modules. They facilitate the greatest degree of re-use, most clearly articulate what they ought to be used for, and stateless things are, in general, more composable than stateful things. There is no better unit of re-use than a tiny, pure function.
JS code is the most disposable piece of any infrastructure. In all companies(mine included) that I know of, npm and the JS jenga tower of hell is the most brittle element that breaks every fucking day. It's the constant pain you can count on being around.
The stack and the dependencies are a moving target. Like.. every minute. If you had coded in any other language other then JS you would know that.
Come to me after 5 years and tell me if your "professional" JS code you're writing today is used by anyone and then we'll talk.
I'll be waiting.
And i've coded plenty in other languages. None with anywhere near as good an experience as NPM if you know how to use it properly.
Don't worry, we ain't going nowhere. Think hard and take as much time as you want.
Pure functions are indeed a good target for modularity and deserve proper documention and proper testing, at the very least.
For the readers of this comment:
* How many functions did you not commented last time you wrote some code?
* How many functions do you leave untested?
* How many functions did you wrote more than once?
* How many "trivial" functions did you wrote that actually took you 3 hours, because it's actually tricky, so you checked other implementations and tried to wrap your mind around it.
Check haskell's hoogle to a small sample of this concept.
You call 4 basic assertions "well-tested"?
What if it was well-typed?
What if it was awesomely documented?
Would that be ok? I'm asking, because a lot of the discussion in this post comes a single simple function being a dependency.
b) The details of this particular project are orthogonal to the philosophy of small modules generally. Whether or not this module is well implemented or well tested has no real relation to whether or not it is a good idea to compose tiny modules.
b) I agree, but you were the one offering "well-testedness" as an argument :)
Don't forget how many different targets there are for JavaScript, even if you had a test environment with limitless budget, it wouldn't compare to having a popular package on NPM.
--John Carmack
This is a pretty weak argument. What is "cohesion" and why do we care that modules have it? Joe Armstrong, one of the creators of Erlang, has argued the opposite (http://erlang.org/pipermail/erlang-questions/2011-May/058768): that lots of small, individual-function modules are better than a "misc" module that grows endlessly and may overlap with other people's "misc" modules.
Calling a function instead of writing the code yourself doesn't mean you've forgotten how to program! The real problem here is the cost and risks associated with dependencies in general (both of which are actually lower for single-function modules), and the broken package removal policies of npm.
(name) =>
switch (name) {
case 'sin':
(x) => sin(x);
break;
case 'cos':
(x) => cos(x);
break;
}A trig package like yours could be implemented on top of sin and cos packages if it's something you find useful.
Stringing APIs together is what actually programming is. This is building software and for instance when i use .toString() method I can easily forget how it is done, focus on other high level things and don't care about dependencies, as long as everything works fine.
Let's admit that the main problem here is with broken npm, rather than packages themselves. If someone has written the "leftpad" function, it is so I don't have to write it again, and I can save probably 15-40 min programming and checking some corner cases.
Also please note that javascript can be really tricky down in the details. So if there's anything that can help, it's better that it exists, rather than not.
It is absurd to have packages suddenly retracted and important parts of the ecosystem stop functioning. This never happened with other languages I have used. Maybe we need a way to make sure the packages are always going to exist. Checksumming and adding the checksum to the version number would be useful too.
This discussion about the "isarray" package is probably my favorite: https://www.reddit.com/r/programming/comments/4bjss2/an_11_l...
Programming, and especially startup programming, is being taken over by people who are primarily technicians rather than engineers. They want to assemble prefab components in standardized ways rather than invent new things. They are plumbers who know how to install from a menu of standard components, rather than civil engineers desigining purpose built one-off aqueducts.
It is the inverse of the "not invented here syndrome." The technician-programmer is trained to minimize time spent thinking about or working on a problem, and to minimize the amount of in-house code that exists. The goal is to seek quick fix solutions in the form of copy/paste from StackOverflow, libraries, and external dependencies to the greatest extent possible.
In house coding should, they believe, ideally be limited to duct-taping together prebuilt 3rd party libraries and services. Those who want to reinvent the wheel are pompous showboating wankers (they believe); creating your own code when you don't absolutely have to is a self-indulgent waste of time for impractical people who just like to show off their hotshot skills and don't care about getting things done. Move fast and break things and all that.
This began with stuff like PHP but really got going with Rails, which preached convention over configuration as a religion, and supplied a standardized framework into which you could easily fit any generic CRUD app that shuttles data between HTML forms and a database (but is painful if you want to deviate from that template in any way.) Note that Rails doesn't use foreign keys and treats the relational database as little more than a glorified persistent hash table.
This set the stage for Node.js (why bother learning more than 1 programming language?) and NoSQL (why bother learning how database schemas work?)
For substance: Consider how this is all part of the effort by management to make programmers feel as interchangeable and insignificant as possible.
And it's not even in the name of quality. Plenty of software out there breaks because of its multiple single points of failure in the form of dependencies.
This then incentivizes new coders to take up one of the popular languages, because that's what most of the advertised jobs are for, creating a self-reinforcing positive feedback loop.
Start thinking about what specialty you want now, today. Do some research, talk to professors and other students, etc. By the time you are a senior, you should know what you want to specialize in and be well on your way.
Then, you can get a job that actually requires deep knowledge. Since you actually care about this, you'll quickly become bored and frustrated if you take a standard startup code monkey job where you are expected to write Rails or PHP code all day, but then that's the only experience you'll have. You'll also be competing with less well educated programmers who are willing to work for less.
you don't need to design a new type of screw to make a one off aquaduct, you don't need to design new bearings to make a gear box, you don't need to design a new opamp to make an amplifier, nor do you need to design a new MVC framework to make a one off CRUD app.
You use off the shelf parts and combine them with your knowledge and skills to produce and effective solution for your constraints. If you can't do it with existing stuff, then you design something new that can do it.
Moreover, the difference is that today's technician-programmers don't know how screws and bearings work at all. The engineer who uses prefab screws and bearings still understands them in detail and knows when to deploy them and when not to. Rather than understanding what bearings are for, the technician-programmer has a problem X, reads a blog post that says, "I solved Y by adding some bearings," and thinks, "gee, X is similar enough to Y that adding some bearings may work."
They add some random bearings, some screws, some cams and driveshafts here and there, without any deep understanding at all of why you ought to use these things in a design... and over time build a Rube Goldberg monolith with 10,000 moving parts sticking out randomly that nobody actually understands at all, so as a result everyone is terrified to work on any part of it for fear of inadvertently breaking the other 60 hidden dependencies of that piece.
The thing that will speak the loudest is actions and results. If people don't like depending on modules, don't. If you do, do. Eventually everything will be lost and forgotten like teardrops in the rain.
If you do have specific short term goals -- such as, say, "building a profitable software product before we run out of money" -- then all methods of approaching the task are demonstrably not equal. Some will work much better than others in terms of achieving that goal.
We can study the empirical results of the numerous attempts already made by other people and thus avoid repeating their mistakes.
I think we can avoid those mistakes but what I see here is people telling other people what to do in a way that sounds unconditional: "x is better regardless of your situation." Some techs are better for x, other for y. Merging someone else's experience tree into your own is good, but it must be contextualized by their perspective--they're working on mobile embedded soft vs. you are on web, maybe, for instance. Or games vs. databasey things. Or their job is a performance engr but you are UI...
'"The Excel development team will never accept it," he said. "You know their motto? 'Find the dependencies -- and eliminate them.' They'll never go for something with so many dependencies."
In-ter-est-ing. I hadn't known that. I guess that explained why Excel had its own C compiler.'
I've experienced the joy of trying to make sure Win32 code runs on Windows 3.11 (with Win32s), Windows 95, Windows 98, Windows Me, NT, 2000 and XP.
Generally it all works, but sure enough there are times when a small code change breaks one or more of those platforms.
Luckily these days things are much better as Win32 is a lot more consistent.
The entire Javascript ecosystem is a huge catastrophe. It will collapse any time soon. It's complex, fragmented and no one really likes it. There are a dozen different tools to get started. No one even understands how to get started easily. There are no fundamental tools. Everything is changing every week. You can't just build a product and then rebuild it even a month later. Nothing works anymore a month later - your dependencies have changed their APIs, your tools have different flags that do different things, there are new data models that you never needed and shouldn't even care about.
The developers are in high stress. Devops engineers are in even higher stress because they get to see what developers don't.
It's a huge mess and my advice to "prefer core-language solutions to small abstractions to small helper libraries to general libraries to frameworks" (http://bit.ly/1UlQzcH) hasn't been more relevant than today.
Software should be developed using least amount of complexity, dependencies, effort and using fundamental tools that have been and will be here for the next 20 years. Cut those dependencies, you don't need them. They're here today and won't be here tomorrow.
function leftPad(str, width, pad = ' ') {
const actualWidth = Math.max(str.length, width);
return `${pad[0].repeat(actualWidth - str.length)}${str}`;
}
It also doesn't work on older browsers - both string.repeat and template strings are new. You could fake it with string addition, but addition behaves oddly in the case your arguments are numerical, whereas template strings handle that. There's also a trick where you can say (new Array(desiredLength + 1)).join(' ') to make a string that is the appropriate length, but you've got OBOEs to worry about if you're not paying attention (Array.join puts the character between the elements, so you need an n+1 array for an n-length string). Also, at least on some browsers, Array.join is pretty cruddy, and you really ought to construct the string with an old-fashioned for loop.
Javascript has all kinds of weird corner cases and lots of browser compatibility problems. The fact that someone's written a decent implementation of something that should have been standard in the String object means I don't have to worry about it.
Of course, I do have to worry about stuff like losing access to left-pad when someone throws an npm tantrum, or dealing with future build issues if npm becomes untrustworthy. A cryptographically sound package manager seems like a reasonable want, especially after this week's issues.
But if your take-away from this whole problem is "meh, javascript devs are lazy", you're missing the point.
As a python developer I would never publish a small package, simply due to the overhead of setting up a PIP package.
Spoken like someone who writes functions in 5 minutes that I find bugs in later.
Just because a problem is simple to describe informally doesn't mean it is simple to implement without bugs.
In Python I would write ´if num > 0´, if it matters that it is an integer I would cast it to int, and that handles everything you should be handling at that level. If your user passes in a list instead of a number type, the code should crash because then there's probably an input error somewhere. If a user has monkeypatched a base class to make a number into something else the code should also fail and you deserve whatever happens to you.
Catching every possible error is a mistake.
I'm fairly old, so I remember the complaints a decade or two ago that people had where "We can compose hardware from IC's and you don't have to know what's going on inside and it's all standard and just works! Why can we not do that with software?!?! (of course that ended up with things like CORBA and DCOM, which was all wrong)"
aaaand here we are in a situation where code re-use is actually happening on a wide scale and now you're complainig about that?
28k lines in an empty project? ha, how many lines of code does the preprocessor generate for #include <stdio.h> I haven't actually measured, but I bet it isn't that far off from 28k lines.
Granted, these modules don't do such useful things, but that's the environment their creators were immersed in.
It takes more than two minutes. That little module has a test suite, if you're including it then you have some assurance it does what it says it does. If you write it you've got to worry about whether it works or not.
Seems like a lot of these basic Javascript functions need to be built in to javascript/node itself or consolidated down to a single package of common core functions that extend Javascript. Like padding, is array, etc. As others mentioned, these are fundamental things in other languages.
Yes, these are simple tasks but do we gain anything doing these simple tasks ourselves? I find there is a finite amount of focus I have when programming, and I'd rather spend that solving the bigger problem.
This reminds me of a discussion I overheard between two professors when I was an undergrad. Prof 1 "This new language makes it so much easier to x, y and z" Prof 2 "Yes but what's the point, I can do all these things in Java" Prof 1 "I could do all these things in NAND gates, but I'll get more work done if I have this tool."
DEPRECATED. This is a rather silly package that I do not recommend using. It's easier to copy the ~20 lines of code of this package and customize that code, rather than downloading and learning how to use this package.
This mess appears to have convinced the author of that.
https://www.npmjs.com/package/babel-code-frame
And the fix was just to drop the functionality and inline the function as a no-op:
https://github.com/babel/babel/commit/09287643c712bcd203bbd6...
I use lodash in almost every javascript project I start, big or small because it makes my life easier.
I'd rather use the lodash isArray than roll my own.
I think it's totally fine. Like other people said, it's the mindset we borrow from Unix, do one thing and do one thing well. The function would be well tested, and could be reusable.
I don't understand why so many people just require lodash into their project (when they start project) while they only use one or only minimum set of the functions. I mean lodash is a very great library with clean and well tested code, but it's also quite bulky like a big utility lib, and for me most of the time I only need one or two of the functions I would just go to npm and find a module just do that thing.
If I can spend half a day writing the code myself, I will -- because I know it will prevent headaches down the road. Yes, yes, I know any code I write adds the probability of bugs down the road. But at least the roadblock would be of my own doing, and not dependent on if/when the package maintainer upgrades their stuff.
Back in the day we would have built up various pieces of library code, which we would have utilised rather than continually guessing best practices. For example, the isArray method cited may be trivial but is also non obvious. We'd probably have something like that in our library of useful snippets.
Sometimes we may have shared the code on forums and the like and people would copy and paste the code, sometimes into their own libraries. We would locate them by browsing known locations or inefficiently querying search engines
Now we utilise a central resource and simple tools to have a global library of code. Instead of searching far and wide we query a handful of tools that effectively does the copying and pasting for us.
How that can be considered a bad thing is beyond me. It's not a question of knowing how to code, it's a question of using your time effectively.
Granted, there is the problem of so-called modules being removed and dependencies breaking. This can be alleviated by vendoring your modules, a simple task with most dependency management tools.
Personally I think that published modules should persistent indefinitely based on the license the code utilises, although I'm not clear on the actual legalities of the recent npm issue (although if it's due to a trademark complaint, I don't see how it would ever be enforceable for completely unrelated code in any slightly sane country).
Maybe we've forgotten how to /just/ program. Everyone bangs the drum so hard of "let github be your resume." Incentivizing putting every brain fart you ever had out into the universe instead of just keeping it to yourself.
Just a thought.
Just look at the whole build system and module hell of C and C++...
I totally like the language C++ but I hate the tooling around it with a passion. As long as you only need stuff from stdlib then you are fine, but as soon as you want to create some custom library for use in your other projects? Which should compile on Linux and Windows? Either you use make and Visual Studio solutions or you have to fight with cmake... Just thinking about it makes me angry.
For this reason I just don't use C++ very often. It is way easier to just use Python because most stuff is just an 'import library' away and you can concentrate on your actual program instead of fighting against the build system.
But god forbid you actually try to ship your Python program to your customers...
Why do I as a programmer have to deal with all this crap? I want to focus on programming and not on library management amd writing make files. My CPU is idle 95% of the time so there is enough processing power which could solve these problems.
I have high hopes for Rust and it's integrated build/module system!
The problem with proper unix tools, the ones that do one thing well and integrate with other programs as a part of their design, is that they make doing cross platform work a problem, because those other programs might not exist on other platforms. GNU Make works just fine on Windows, but you're not using just make when you write a makefile. Makefiles typically depend on GNU Make, pkg-config, gcc/clang (or another compiler with the same CLI), coreutils, bourne shell, and an assortment of a few other utilities. That means that Makefiles by design will only work on a single platform.
The other operating systems have essentially made themselves into the same platform for these purposes through the use of system package managers. I've heard Windows is kind of getting one? More like homebrew than what we have in linux/bsd-land but that might still help. But maybe Windows developers will still be reluctant to install software with MinGW dependencies. That would be unfortunate, but fair; I wouldn't install a package that had Wine/Mono dependencies just because some asshole wanted to use their Windows build system on Linux! Hell, I avoid installing smaller things with a Qt or GTK3 dependency simply because none of the software I use needs those and I don't want to clutter my installed-packages list for something that isn't important.
I'm not a huge fan of Rust's Cargo. The list of requirements that the Rust team had for their package manager and build system made sense, and if I had to design a tool that met those requirements, it probably would have looked quite a bit like Cargo, but it makes me sad that the most reasonable design is that which abandons the unix-like simplicity of the python and C (and perl?) toolchains.
1) provide a single, amalgamated .h/.cpp pair 2) provide per-platform, per-config precompiled static and/or dynamic libs 3) provide all dependencies in the form of 1 or 2
Voila! Your goal should be to provide something that can be trivially added to an existing pipeline.
No, the problem is that there is no standardized way to deal with modules and libraries in C and C++. If it would be in the standard the cross-plattform support would be much better.
> the unix-like simplicity of the python and C (and perl?) toolchains.
For small C projects the make/autotools system might be "simple", but for larger projects? It is horrible.
We haven't forgotten how to program. We've got better at it.
While I agree, there are two counter arguments:
1) As noted in other comments, this is a reflection of the core libs of javascript not covering enough of the basics. This is a subjective thing when it comes to where to draw the line, but wherever the border lies between "core" and "community" libs, on the other side you start running into things like "L-pad" and "Lib-L-Pad" and "L-pad2". If there's a great enough fundamental need, you experience a lot of crowding where lots of people offer up their solution to this, and reconciling that across larger dep. chains that disagree on which one to use can become a real burden.
2) Have you ever had the conversation with an auditor (PCI, HIPAA, etc) that your application is actually built on top of a huge substrate of software written by unaffiliated 3rd parties? And that between your own iterations you could easily have different versions for any/all of them? It's a difficult conversation. Much less the explanations to QA about why a build failed because lib X was updated to a new hotfix version in the 14 hours since yesterday's build, after a couple hours of wasted time of initially suspecting your own diffs, and trying to navigate through all the indirection between the actual stack trace and what actually caused the blow-up...
If there's a flaw to this debacle, it's that packages can be un-published. That is some grade A+ BS.
But no, there is no such thing as a package too small. Coding is hard. Collaboration should be default-on, not default-off.
The loudest people in the Node community have been evangelizing this practice for as long as I can remember. This shouldn't come as a surprise.
The argument, "If I didn't write it I don't have to think about it" is ludicrous. I just have to point at the left-pad incident disprove the premise of this argument.
The analogy of building things with a bunch of npm lego blocks is laughable. Those responsible for advocating the use of trivial functions by acquiring module dependencies are leading the masses astray.
"But, If I find that there's a bug in a module I can AUTOMATICALLY fix it everywhere!"
No.
You still need to assess how the change to that module impacts any code that depends on it. Just by updating a module and posting a "minor" bug fix can lead to other bugs that RELIED on the behavior as it was originally written.
It's simple, write your own trivial functions. Test them. Maintain them.
P.S.
Another module that can easily be in-lined to every code base you own. (3 million downloads this week).
How anyone can deal with JavaScript for more than 5 minutes is absolutely beyond me
If you're writing only for modern browsers, you don't need it (if you actually visit the code in question you'll see it defaults to Array.isArray - in that sense, it's a polyfill or whatever). But if your code might run on old browsers, it can't hurt to have it, and it cleanly encapsulates a trick that you no longer have to remember the syntax for.
I even consider PHP a "more sane" language because you at least have most of the useful utility functions in a global namespace and everyone uses them. Of course, the real ideal on this is Python's solution: a nice set of standard libraries that you know are baked in, but most of them you still import explicitly - hence it's pretty easy to write even large Python applications that have a small and comprehensible number of dependencies!
(And more generally: our strike for "more efficiency" in programming is stupid imho! I'd always take a less efficient solution, even "less safe/tested", if it's more "understandable" and "explainable" and sometimes, paradoxically, making things a bit more monolithic and centrally planned makes then orders of magnitude easier to reason about for our tiny ape brains...)
In fact this should be the key selling point for stdlib to JavaScript. It actually saves money from their employers if trivial functions are mostly in the same place.
However pushing a stdlib is difficult there is a lot of hate for Python's "kitchen sink" library in JavaScript world which is totally misplaced. It maybe founded on fact that you should then minify the whole package to use one function in it, but this could be solved with dead code elimination (with proper type checking) in the minifier.
Is there? Most "js people" have never had the pleasureful experience of working with a "pleasantly opinionated" language like Python and the similar feel of its libraries, even the 3rd party ones.
Really, there's something really awesome in the Python ecosystem that should be spread to others before node takes over the world: If done right, being opinionated ("one way to do it") and centrally planned (a BDFL or smth like that) is A GOOD THING for an open-source language/infrastructure project! And it's also good if you extend the philosophy to the libraries and frameworks. Don't ditch the baby with the bathwater!
But I have a really hard time translating this message to "node-speak" unfortunately... Thanks for the "saves money" tip though, never thought of that, will try it ;)
You would be surprised of how many developer these days have 'afraid' to write such functions, or how lazy they are, they found this thing and just add to a project, then push to some google list and the project got a lot of followers, and in the next day the project has changed about 90%. I saw this happen over and over again in this ecosystem, this is insane dude.
A lesson I learn is: you _need_ to read every module source code before add to any project, the NPM ecosystem has so many "shits" out there. You cannot trust in any npm module, recently I tried to trust in a module that has more than 5k stars, but I found a such ugly bug on that, that I feel my soul die, and I swear I hear the angels cry, thats not how open source supposed to be.
These days, seems that people dont care about the 'bug free' as long as it work a half way.
A Java left pad implementing a Padder class instanciated by a PadderFactory that you can get by a PadderInjector from a PadderRegistry all that accepting an AbstractPaddingString.
Then one in Ruby where you monkey patch the string to that it pads, then add an AUTO_PAD settings set to True by default and a fluent API to chain padding.
Then one version PHP version containing pad_string, then real_pad_string that deals with unicode, then real_pad_string_i_swear_this_time that that call str() on parameters automatically.
Then a Haskell one with a monad and a trampolin and some |=> giberrish to mix it all.
Then a Go one that creates a string padding microservice. With a docker file, just in case.
And the last trend in Python, an aio_string_pad which gives you an asyncronous padding coroutine, but under the wood calling run_in_executor, and optionally auto start an event loop.
This is 99 bottles of beer all over again.
In reality it could take an hour to get this working properly, but it does take only a couple minutes to verify that the solution here is correct. There are certainly good reasons for not adding extra dependencies to your project, but trading a known amount of time to check that an existing project does what you want for an unknown amount of time to redo it yourself is probably not a great bet.
Google, authors of V8 and #1 subject matter experts on V8, have published coding standard. Does someone use it in the node community? No. Everyone loves "standard", a lousy standard that allows everyone put a badge on their github page while still having a code base full of vomit.
JSDoc. A great solution for documentation. You would expect major libraries to adopt it, or a similar thing. But again, no. Major libraries such as busboy do not use them. The documentation resembles a napkin.
Then everything else: input validation, error handling, consistency... etc. Take "request" for instance, one of the most widely used libraries. The state machine it implements is inconsistent. You abort a request and get a timeout, you can abort a request without starting it and get an exception. Issues that will drive you insane while debugging.
Express, one of the most widely used web frameworks on node.js. Do this on a route: setTimeout(function(){ throw new Error(); });. Great, now you have broken out of the error handling context. Great job.
Node libraries suck all across the board. It's the PHP of the 21st century. There are exceptions, like: lodash, bluebird, and others.
function repeatString(str, len) {
return Array.apply(null, {
length: len + 1
}).join(str).slice(0, len)
}
[1]: http://stackoverflow.com/questions/202605/repeat-string-java...
This function repeats a string up to a certain length. e.g.
repeatString(foo, 10) = foofoofoof
This could entirely be what the developers needed in their code base.
The fact that it doesn't repeat a string N times, is at best, an example of bad naming.
The issue here is that someone unrelated to the core team grepped the node codebase for the term they wanted and threw a function up as a SO answer without understanding what it did.
Consider the following:
1: var str = 'foo';
2: str += 'bar';
3: str += 'baz';
Now consider the following:
1: var str = ['foo', 'bar', 'baz'].join('');
Once you have a large number of elements, or are joining large strings, the performance difference (especially regarding garbage collection) could matter.
Now of course there are times you'll want to reach for a library, say for something like an http router, and up until recently the dependency management side of Go has been lacking. But when a pattern or library arises that many find useful the core team is open to pulling that in to the standard library if a strong enough case is made, for example in the context package (https://github.com/golang/go/issues/14660).
1. Make an observation about a popular thing. 2. Blindly extrapolate. 3. Make one or more broad, controversial statements. 4. (Optional) Nuance.
That being said, there is something to be said for using these micropackages. Left padding a string is easy, but you might just have forgotten about that one edge case where in browser X and language Y you have to do things different. It's not really the case here, but things that seem simple at first often turn out to be hard because of some edge cases. One might hope these edge cases are solved if they use a library.
As staid and corporate as it might sound initially, it's a very smart thing to do. One screw-up with licenses could be catastrophic. Are you all really checking that carefully?
I can't even imagine how any sort of proper legal checks could be done with a trillion micro libraries.
Why would you want to install a 500kb dependency that has only one function you need, when you can install a 10kb dependency that has it?
Would you want each of your five 20kb dependencies to re-implement the same 5kb function, increasing the code you must send to the client by 20%, or would it be more optimal for each of those dependency to use the same 5kb function?
The author rants about practices of developers from different programming environment, without experience, without figuring how things came to be. If he did give an effort to think from the perspective from Node.JS developers he’d have addressed the previous two points.
This is like going to a friend’s house and complaining everything is put in the wrong place. It would have been wise to immerse in Node.JS conventions and observe for a while before making comment.
EDIT: Reply to scrollaway:
I've also understated the problem.
Let's look at the problem in the current Node.js environment, it's not uncommon for a web app to have 20 dependencies, each of those have 10, and each of those 10 have 5. That's a total of 20 times 10 times 5 = 1000 dependencies in total.
Let's say you were to remove a 10 line library function that's "standard library-like", used by 15% of those dependencies, and have each of the existing dependencies re-implement that in each of those dependencies that uses it.
15% times 1000 times 10 lines is 1500 lines of code.
So if you're going to troll a solid argument by nitpicking, do it properly and get the details right.
Also, have you ever seen a commercial library require >2MB of minified JS? Here's an example:
http://cdn.wijmo.com/jquery.wijmo-pro.all.3.20153.84.min.js
That file is almost 2MB minified (5.6MB unminified).
And that is is only 1 of the few different files required for our dependency on it, there is a bunch more javascript/css also required. Would I use that library on a new product? I'd probably advise against it due to the size, but a legacy product has dependencies and you don't have the freedom to rewrite everything all the time.
> So if you're going to troll a solid argument by nitpicking, do it properly and get the details right.
First of all, I don't appreciate you calling me a troll. Someone who mentions "5kb functions" clearly has no idea of what 5kb represents, period.
And second, this is not a solid argument at all. There is logic behind it, which is why we usually have high quality stdlibs in popular languages. Javascript lacks that. So instead, javascript gets this absolute mess of an ecosystem, where things like this can happen.
Several people have brought up various issues with the way it's done now. Dependency hell, for one. Lack of discoverability, which in turn leads to duplicate libraries, which in turn leads to a system where you have 1000 dependencies, but despite your "modular" idealism you still have hundreds of duplicate dependencies. Not to mention all the duplicates of different versions of the same dependency.
This "saving lines of code" math is completely broken exactly because this stuff is not in the stdlib. The various issues with the JS ecosystem mean that the actual results are nowhere close to ideal, and have a net negative impact.
I also love when people mention how much "one time cost" is saved and back it up with similar math, completely forgetting the amount of continuous time wasted downloading and installing these dependencies every time.
You are a troll because talking about the actual number I used and how good a grasp I have in my mind completely misses the point. You're criticising the javascript ecosystem, I get that, but you completely missed the point of my comment. If you're not going to try to understand how Node.js conventions came to be, what it's strengths and weaknesses are, what the trade-offs made were, then it's inappropriate to focus on particular weaknesses of its practices.
It's like going on about how Objective-C development sucks because it's hard to do on Linux. But you're missing the point about what Objective-C's strengths are that attracted Objective-C developers in the first place.
The author should spend some time acclimating to Node.js best practices before writing this article. And you should do the same before you start knocking down straw men to prove how right you are.
I agree with https://news.ycombinator.com/item?id=11313092
You can dish it out, but you can't take it.
From the perspective of Babelify users, a major bug was introduced into software they depended on. I don't know how much money in developer time was lost due to this but it would almost certainly be in the thousands of dollars.
And it could have been a lot worse. It could have been something more complicated than left-pad. The author could have introduced a vulnerability or outright malicious code, or been hacked and done the same, and millions of people would have downloaded it and run it.
Arguably, small modules are good if you control them. Maybe they are more composable, maybe they enable better testing, maybe they encourage code reuse. I am not going to argue for our against small modules.
But none of the positives of small modules matter if an unknown developer who you have no reason to trust can change or unpublish the module out from under you. It's irresponsible as developers to risk our employers' and clients' businesses in this way for a function we could write in five minutes.
If jdalton declared a jihad on arrays tomorrow and decided to pull all array related functions from lodash, we would have the exact same problem.
If kriskowal decided that Q must mirror built in Promise and published a version that does this tomorrow, we would again have the exact same problem.
There is only one connection between this problem and the small module approach. As the size of a module decreases, the number of dependencies increases and so does the number of authors that produced your dependencies. With the number of authors increasing, the chances that some author decides to go rouge or protest for some reason also significantly increases.
Therefore, its irresponsible to use this approach with a package manager that allows an old, established module with many dependents to be unpublished so easily by the original author.
The problem with unpublishing is that it changes an existing version. If instead of depublishing a correctly versioned empty module named leftpad was pushed to npm (increment major, because a non-implementation is incompatible with an implementation), there would not be half as much pain.
As long as unpublishing exists, micromodules increase the "attack surface" to this specific method of changing the contents of a specific published version.
1. The developer, 2. The content distributor, 3. The code (by git SHA ref perhaps), 4. The contract of the code (using formal verification), 5. The legal contract with the code supplier.
And if you trust 1, do you expect of him to sign the stable releases using GPG tags?
All the focus on OSS nowadays seems to be on 1, but as professional engineers, shouldn't we focus more on 4 and 5?
He is talking about learning to code vs asking google for implementations of even the most trivial things and about what a useful library is.
http://erlang.org/pipermail/erlang-questions/2011-May/058768...
npm actively encourages structuring projects as many tiny individual modules and dealing with the resultant dependency trees and deduplication. Both of these things (along with the ease of publication) combine to encourage people to share their packages.
They make it incredibly easy to consume code from other people, but but at the same time provide a similarly low-barrier mechanism to retroactively change published code. That combination seems like a way more deserving topic of criticism than the unending refrain of "developers these days are so lazy".
http://erlang.org/pipermail/erlang-questions/2011-May/058768...
Maybe the unit of modularity should be a single function and we can do away with modules?
No need to have global dependencies on small snippets that really should be in a core library anyway. C has libc, Java and C# have the creator's (Oracle or Microsoft) standard set of libraries, Python has the "batteries included" stuff in all the different distros. And so on. All of these snippets rightly belong elsewhere, not in packages.
And even if you did get them added to the right libraries, I guarantee you that you will not get rid of the need for a collection of small, and somewhat random, in-house functions, classes and libraries.
Write your own goddamn utility classes, people. Or learn how to suspend package releases, include them in your projects, and smoke test your releases.
http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm
I may have just invented a rule for choosing programming language and systems there.
In languages with linkers, the better linkers would discard all unreachable functions. Then came DLLs. Use one function in a DLL/.so, and the whole thing has to be loaded. Go and Rust are usually statically linked, reflecting the fact that pulling in big DLLs was usually a lose. You rarely have two different programs on the same machine using the same DLLs, except for some standard low-level ones such as the C library.
In my opinion it will not be done right in 80% of cases.
Every breaking change requires updating the major versions, but developer are hesitating to go from 1.0.0 to 6.0.0 in a month.
The way out is staying in the 0.x range therefore abandoning semver alltogether.
A nice write up about how packages are not following semver in java-land:
http://avandeursen.com/2014/10/09/semantic-versioning-in-mav...
My app might well have an is-positive-integer function but it will include a range of context dependent choices about e.g. floating point, infinities, zero, null, "9", "9 ", "09", boxed numbers, string representations exceeding js max int, etc. etc.
I firmly believe that building on what has already been done allows for much safer code written at a quicker pace. Why should we constantly repeat ourselves? Also by using npm modules we can abstract logic and prevent someone on the team from going in and modifying it for their own use. It is a documented/constant function that we can use knowing exactly what it does. Is it a better world where everyone just copies code out other's repos and then has to include the licence/docs/tests along with it? It's much easier to just pull in the repo which contains everything and make it trivial to see where a function came from.
People are blowing this whole thing way out of proportion just because it makes a good headline "11 lines of code broke node"... You can all try to shame people who build on what's come before and chant "Not invented here" but I'll opt to build on what is proven to work instead of rewriting everything. At the end of the day that's what ships products.
That being said, I think this is a perfect example of where a good concept (small, tightly scoped modules) is applied dogmatically at the cost of the codebase. It's the node.js equivalent of AbstractRequestFactoryFactoryFactory stuff you see in Java, and the Mock Messes you see in Ruby.
Clearly to mitigate such a tightly coupled dependency, Left-Pad should be a micro-service. :-\
Save these for the things you can not do in house, like NaCL. Don't write that yourself.
But string padding, sorry. Any developer worth their salt would laugh at adding a dependency for this. It's irresponsible, and comes across as amateur hour.
This is a simple case of optimizing for responsibility. The inexperienced programmer does not know how to do this, because they have not spent enough time being responsible, and having to deal with the fallout of managing responsibility poorly.
An experienced programmer carefully manages responsibility. A simple function, that is easy to understand, easy to test, and easy to reason about, is something that makes more sense to either pull all the way into your codebase, or write yourself.
Years of doing this means that managing dependencies should never be just slap it into your packaging system and start using the function. If you are on the line for making the wheels turn for a large scale platform that is directly connected to a monetary transaction of some nature, you will quickly find yourself preferring to remain responsible for everything that you possibly can control. There is certainly enough that you can't control to keep you on your toes.
Nothing is included. And that's a feature. The web is not trying to be the kitchen sink. That's iOS. They provide high level APIs for everything. And as a result, the platform is architecturally only as vibrant as Apple can make it.
Now maybe you're happy with Apple, maybe you love the iOS APIs. But if you don't, you're stuck. There's not a rich bed of alternative view layers that you can draw from to build your own vision of how software should work.
Node and the web browser strive to be lowest common denominators. They provide just the very basics: a document format, a very simple programming language, and an http server. The rest is up to you.
That's pretty scary, and so the JavaScript world has dabbled in frameworks. In the end all-inclusive frameworks are antithetical to the spirit I'm talking about, so things trend towards small modules that do one thing pretty well. People go overboard sometimes. I would argue left-pad should just be a copy-pasted snippet rather than module. But that's not a sickness in the community, it's just developers feeling out how far to go.
If you like every application to look the same, and you don't mind being chained to enormous and restrictive standard libraries and frameworks, then you will hate JavaScript. If you like writing software from scratch, forming opinions about all of the different parts of the system, and allowing each application to be built out of different parts, out of the right parts, then you should give JavaScript a closer look.
Yes, downloading a giant JS lib for one function is insane, hence the ton of tiny dependencies.
However, it is equally insane that basic SDK expectations found in every other language has yet to come to be pre-implemented by the JS engine in the web browser itself. Some basic code ought to already be on the localhost the moment your app arrives.
When the developers of such a serious library as React start to depend on a third-party one-function module made by some Dick-from-a-mountain (a russian idiom that means a random person who did nothing significant but tries to show out in all possible means), that means React developers are even more to blame than that Dick-from-a-mountain himself.
If you make any really popular piece of software, you absolutely must have a failover plan. No excuses for not having it.
But what's even sadder is that this issue had spawned a new wave of pseudo-elitist attacks on the entire JS dev community. Calm down guys, things like that could have happened for any language that has a widely-used centralized package system (Perl's CPAN, Python's pip, Ruby's gems etc).
Let me repeat that again: languages don't make software bad, people do. Just don't let such Dicks-from-a-mountain rule over your own modules with elementary stuff like leftpad, and you'll be safe.
Sure there was some argument about Unix philosophy, small module doing one thing and does it very well. Did anyone bother considering the quality of most NPM packages? Quality is not reflected with passed Travis CI or extensive community testing and feedbacks. Not at all. Look at the those packages on apt-get. They are modular and robust. They do what they were supposed to do.
Now take a long hard look at the state of NPM. What do we have? People clamoring for reusability and whatnots. Most of them don't even know what they're talking about, just reciting the latest hip statement from the Internet. Being mature about development means accountability for what you do, not pushing shit around that you don't even have knowledge off. As a self-proclaimed polyglot, I love JavaScript as a language but not the ecosystem. It's like watching a dog chasing its tails:
- Endless loops of discussion that help stroke the egos but not improve anything else.
- Craps for resume/repository padding, not for actual developers to use.
- Bandwagon mentality that just pushes the latest fad along, and the herd towards the cliff.
The notion that JS developers are kids playing grown-up, has been reinforced with this NPM incident. If we want to discard that notion slowly, we need to be more mature developers. It's that simple. Here's what I think we could do: - Have a clear idea on what dependency you need. Browser, IDE, terminal etc are dependencies. Basic type checking is not. - Be better craftsmen. Roll and maintain your own toolboxes. Only share a working hammer, not a broken nail or a wood chip. - Note that for each package you publish, thousands more hours would be spent on learning, adapting, using and reporting mistakes. Collectively, the current community wastes so much time with finding the right things to use. Often times, we learn much more by playing with code, even posting on StackOverflow. That's hands-on, `npm i` is not. - Own the code better. The idea that teams like Babel and React devs with all the brilliant developers choose to put their eggs in a private corp's whims is just scary. You can't hope to build robust software while playing Jenga tower.
I wrote my own left-pad for a project I'm working now and I had to revisit a few times for tiny problems and lack of time to write tests. I would definitely use `left-pad` module if I knew the existence at that time.
Putting on your devops hat, whatever your dependencies, from a reliability and reproducibility point of view, you should control your reliance on unexpected decisions of npm or third-party developers. A lot of the panic with npm issues comes from people blindly using the npm registry and then seeing breakages, with no safety net. I hate to say "I told you so" but this is an issue we worried about a lot when considering Node productionization last year: https://medium.com/@ojoshe/fast-reproducible-node-builds-c02...
* If you're in favor of the micro-module approach, you shouldn't be relying directly on NPM, and should have something like Sinopia in place. After all, external code isn't the only thing you're vendoring, right?
* Micro modules are fine - but your application code should depend on a privately published utils module whose entry point is a prebuilt distribution of all your external micro-modules exposed through a facade. Your utils module deps are all installed as dev dependencies to avoid the Fractal Nightmare.
* Yay, now you have your own 'standard library' which still manages to leverage the NPM philosophy of distributed code. And if some twit decides to throw a tantrum, it will only impact future builds of your custom std lib - and you'll know about it at build time.
My point being, if something is a completely reusable and basic feature, a dependency is totally just. I remember a few years ago when I and all devs I knew (which weren't many, I was maybe 17) had our own libraries to include in all personal projects we made. It contained features we had to look up once and from then on just automated and imported, stuff like password hashing or input checking. This went out of style as single-programmer programs are going out of style, but the little useful features are still there.
I say this because I strongly believe that reinventing the wheel is unnecessary and can bring more problems than not.
There are many examples, and I could come up with a made up one, but here's a very real bug that I debugged from another programmer not so long ago:
So, he came up with a JNI for SPSS's C library, applied it correctly, and got haunted for lots of months with an unsolvable bug. The problem? he miswrote the final copy of the file, and sometimes, some bytes where copied twice.
He tried to solve this problem for a long time (and eventually lived with it because SPSS was still resilient to this)
Is this concrete example of a 'module' ridiculously short? yes, but my logic still holds IMO.
It's not that we can't write a left pad function ourselves. It's that we might easily miss an edge case or make a mistake in doing so.
The author seems to be hung up on a preconceived idea of what a package "should" be without actually offering a compelling argument for why a single short function can't be a module.
Yes, every dependency you introduce is a liability. But so is every line of code you write. I'd much rather take the risk on a shared library which can be audited and battle tested by the entire community.
If a function is so easy to write, it's trivial to check out the module's code, review it does the right thing, and then lock the dependency.
It's good to have small packages. Don't forget that the underlying ECMA script is changing so the implementation of these 'libraries' are (or will be) different over time from what they used to be. If somebody finds a faster way to do the method, then it will be done.
Finally, anybody who has used js in the real world understands how many corner cases there are and how difficult it is to make durable methods (i.e. how to know if an array is empty - which requires like 4 different conditions).
Making this about is-positive-integer misses the point that this is a social/political problem not a technical one. A language ecosystem must address concerns of business continuity as first class concerns.
I've got two dependencies besides the basic framework my project is built on: A scheduling engine, and a database interface. I eventually hope to factor out both.
I mean, I can certainly see using more dependencies than I'm using. Mail is a great example of something that probably should be handled by dedicated, well-tested code. But to carelessly dump piles of third-party dependencies in your project to save a few minutes, as talked about in this article... just crazy.
Relying on reimplementation, copy-paste, npm shrinkwrap, or various other ways of importing third-party code into your repository results in the following advantages:
1. You know exactly what goes into your product, and can audit everything for security, performance, or coding standards.
2. You often end up importing less, as third-party modules may have functionality that you don't need but other clients do.
3. You can modify the resulting code to add showstopper functionality, even if upstream doesn't want to.
4. You aren't subject to the whims of someone removing your dependency from the Internet or replacing it with a version that does something you don't want.
Relying on lots of little libraries installed via package manager gives you the following advantages:
1. You can easily install & try out modules that other people have written, letting you test out new features on users more quickly.
2. You can share code with other modules that have the same dependencies, often reducing the overall size of your system. This is important when there's a cost (eg. download size) to your total bundle.
3. You have less code for your engineers to read & maintain.
4. You can easily track licensing & contact information for your dependencies.
5. You automatically get any new features released by your upstream dependencies.
6. You automatically get security updates and performance enhancements released by your upstream dependencies.
The last is nothing to scoff at: imagine if the headline, instead of 'left-pad breaks the Internet!', had been a security vulnerability in left-pad which literally broke the Internet. Imagine how hard that would be to fix if everyone had copy/pasted the code or re-implemented it. This is not an academic scenario either: remember "Nearly all binary searches and mergesorts are broken", published by the guy who wrote the broken binary search implementation in the Java standard libraries?
http://googleresearch.blogspot.com/2006/06/extra-extra-read-...
Always copying your dependencies into your source tree is not the answer to this, no more than always relying on npm modules was the answer to updating your dependencies. They both have pluses and minuses, and if you really want to be a good programmer, you need to weigh both of them. For my projects, I tend to use whatever libraries I need when building them out (via npm, if possible), and then periodically audit the dependencies to make sure I'm still using them and they wouldn't be better off incorporated directly into the project. I wish more products did this, but I don't control what other programmers do.
npm install <small-dependency> <destination> --save-inline
When I installed a simple lines-of-code counting tool through Macports the other day I accidentally opened the door to dependency hell as gigabytes of not even remotely related stuff started to build [1].
Without a doubt something is going very wrong with Free Software and package managers. On the other hand, never look a gift horse in the mouth so I may not even be the right guy to complain here.
I'm hoping that proper support for ES6 modules to enable tree-shaking bundle builds (in the short term), and HTTP2 with client support for modules (in the long term), will allow us to head towards a world where we converge around a handful of large utility libraries.
In theory, tiny dependencies was supposed to allow us to only include code we actually needed in our bundled code. Bu the reality is that everyone uses different tiny dependencies for solving the same problem. So you end up with enormous bundles made up of different solutions to the same tiny problems.
Professional racers don't need an automatic transmission, but it's quite helpful for an unprofessional driver.
[1] For the pedantic, it probably was not literally the first thing; I'm sure some folks went to the rest room, others ate something, still others took a nap.
What is it about the node community that triggered this misunderstanding of package management and code reuse?
As for the issue with such a short piece of code being reused by many, why score on number of lines? If it works and is a useful function is more important to me. I am not familiar with bundling within the usage the article covers but we tend to bundle like functions together and the compiler drops unused ones
BTW, isn't that a Facebook project, so aren't they supposed to use a CI? ;P
I'm not convinced its worth it compared to the simplicity of commonjs and module.exports. You have to pull in babel, which has over 40k files to do all this.
Why are people destroying the beautiful simplicity that is javascript? Can those people please go back to java?
Every project I've seen that uses npm always required 100s or 1000s of dependencies.
If building or running your project requires something and you can't explain why, I think there's a problem.
On the other hand, there's a lot of bad practices disguised as "simple" and "efficient." Using NPM for one line functions is a great example of this.
[0] https://github.com/facebook/react/blob/master/package.json
Small packages done right, well tested = maintenable, reusable, stable code.
The problem does NOT comes from packages. The problem comes from un-publishing public packages and centralized repository server.
I have a java project with a lot of dependencies. Does it mean it's bad ? No, but if maven repos are closing tomorrow, my project will not build as well.
Nowadays you use NPM search instead of Google search
The fact is that lazy programmers are lazy. The methods change but the principle remains the same. In 90s people typed in code from magazines and books.
Part of the problem, in my opinion, is that packages are not subject to any review process. These dependency chains get out of hand because they are rarely top-of-mind.
leftpad = (str, len, pd = ' ') => Array(len > str.length ? 1+len-str.length : 0).join(pd) + str
WTF are you talking about? Making this into a module?!
Should tests also be rewritten or copied each time you do this?
What do you do when you've used it in multiple projects and you find a bug or performance issue?
function foo(arr) {
var str = "";
var leftpad = require("leftpad");
for(var i=0; i<arr.length; i++) str += leftpad(arr[i]);
return str;
}The usual complains about many dependencies are mostly void (it's not bloat if you only depend on single functions you actually use).
No... "appropriately sized modules are easy to reason about"
In this case... "Appropriate" has gone out of the window!
Any modern standrards-compliant implementation of JS (following ES5 and especially ES6/ES2015 standard) already has everything that a sane programmer would ever need.
The problem is the lack of a test culture.
We haven't. React developers probably have.
First: why small modules are bad. Lots of dependencies complicate your build, and you end up with the dreaded diamond dependency issue. Failures in a dependency become more likely to affect you. It gets you in the habit of using prebuilt modules even if maybe it's not quite what you need and it would have been better to write yourself. With `npm` specifically, we've seen how its mutability can break the build, though that's about `npm` and not the idea necessarily.
I think most software developers' gut responses are that something is wrong and crazy in the npm ecosystem.
That said, there are benefits that this blog post and others aren't mentioning, related to the javascript situation specifically.
The first one is that javascript is a surprisingly difficult language to get right. Sure, the final solution is only a few lines, but which lines are hard. You have to navigate the mindfield that are is V8 in nodejs, v8 in chrome, spidermonkey, chakra, etc. I've had code work in Chrome before but blow up in IE, and it's really hard to track down and test.
The comments in the blog post are illustrative:
One line of code package:
return toString.call(arr) == '[object Array]';
(testvar.constructor === Array)
The constructor comparison will fail if the array comes from a different context (window).
The other thing that people don't bring up, is that there's not much JS standard library, and in the browser context you have to send all your code to the front end.
So maybe you write these 11 lines yourself, and then another package writes these 11 lines, and another... it adds up. But if everyone uses the same package, the code only gets sent once and they all share it.
Lastly, people talk about how `sin` should be a part of a "trigonometry" package and not by itself. Well, again you're faced with sending a bunch of unnecessary code to the frontend. With webpack2 and tree shaking, or e.g. Google's Closure compiler, it can strip out dead code and so this issue will go away in the future, but we're not quite there yet. So package authors still bundle all these things separately.
So pros and cons.
Sure in some ways NPM has packages that don't deserve the title of a package, but isn't the convenience of not having to reinvent every code worth it?
The comonJS module pattern also helps tremendously, as your requires are block/function scoped. It makes it trivial to refactor (remove/rewrite) code.
Real developers do not do stuff like this.
"most optimal"?
In this test case it comes out slower: https://jsperf.com/left-pad-arrays/3
But ultimately it doesn't matter in the least for 99.9% of use cases:
https://blog.codinghorror.com/the-sad-tragedy-of-micro-optim...
I just tried in NodeJS, you could also just instantiate an Array of a given length and join it with spaces, like this.
var x = new Array(5);x[4] = "abc";x.join(" ");
No idea if it could be faster - but the test you linked to pushes spaces into the Array.
"Immutability at the centralized authority level and more decentralization of package distribution is the solution, not 'write more functions yourself'."
what the fuck does that mean?? they just don't give up, do they... Fucking retards.
The Programmer's Stone first essay is actual as it never been before.
Actually, it is a common pattern. When some activity becomes popular due to very low barrier to enter it will end up in a such kind of mess. It seems like nowadays everyone is either a programmer or a scientist and researcher.
This is the quality of their software and research.
There has been a reason why good schools taught principles (algorithms and data structures) not particulars (objects and classes). But since MIT and Berkeley dropped Scheme-based courses in favor of "pragmatic" Python (thank god not JavaScript) based courses we are heading to a disaster. Java madness taught us nothing.
History is full of examples where assault by mediocrity ruined the whole branches of philosophy, arts and crafts. Instead we have fastfood, mass media, social media and now this mass coding, which combines worst from mass and social.
Just try to compare things like Smalltalk or Plan9 or R4RS Scheme or Zeta LISP of Symbolics or with this stuff.
But seriously this is stupid. Programming shouldn't be the goal. Just because you can write a function doesn't mean you should. Every line of code you write is overhead that must be tested and maintained. I guarantee that if the author chose to hand roll code instead of using packages he'd have a lot more bugs. But he wouldn't know that until he hit some mundane edge case scenario in production.
The problem is, that modules doesn't solve problems magically. Every dependency should be tested, maintained and kept online as well, by somebody else, whom you have no control of. In turn, you have a bunch of black boxes that can blow up your application anytime. Of course, another extreme of write everything yourself is also bad, so it's a trade-off that must be considered carefully. I've build many systems during my career and one of the biggest nightmare I encounter is managing dependencies.
Wait -- code reuse is bad now??
module.exports = leftpad;
function leftpad (str, len, ch) {
str = String(str);
var i = -1;
if (!ch && ch !== 0) ch = ' ';
len = len - str.length;
while (++i < len) {
str = ch + str;
}
return str;
}
Calculating needed length. Using repeat and just concatenating 2 strings would be faster.
I'd probably get 2 of those wrong in some weird way, but I blame javascript. I mean without Google of course.
Good programmers understand the risks of making your system depend on something you don't have control really well; They know how keeping system complexity low is like an good investment which makes your life later easier (low maintaining costs).
Bad programmers stacks up technical debts such as including unnecessary dependencies until the system no longer works.
For example, the quality of your own code.
"Packages", "modules", "functions", and other words can mean different things within different contexts. Well-known and tested functions are useful. Putting them in a "package" is just a consequence of how node/npm work. There should certainly be a much, much better implementation of code sharing, but sharing and using well-known, singular functions should be exactly what we are going for.
Cracked me up :D
Eventually, given a pure enough language, every "package" could contain only a single function, and every function in a project could be published as an independently reusable unit.