CoreUtils implemented in pure JavaScript
github.com
github.com
Oh boy, another one of these. Cygwin is useful because it provides a POSIX C API on Windows. It is not useful because of its shell and coreutils. And saying it uses "pure ES6" is stretching it - it's using the plain ES5 Node.js require/export patterns instead of ES6 modules.
>1/15th of the size [of Cygwin]
Haha, and 1/100th the functionality
Also... Node... Huh? seriously? replace a siongle DLL with Node.JS is simplifying it??
Cygwin is overkill unless you are trying to mimic your exact non-Windows environment, and it doesn't mix well with Windows' vanilla CMD environment. For those of us who want to be working with more Windows than UNIX, but who are used to standard coreutil commands, 1/100th the functionality of cygwin actually sounds kind of great!
Not saying that the commands need to be written in JS, but it seems like every language gets at least one coreutils implementation, so why not?
So much negativity around here. Honestly I find it pretty neat that JS has come this far.
I don't even like javascript.
- it works on my laptop
- but I already have it
EDIT: I've said it on other threads- I like the creativity and exuberance in the JS community. I'm not putting it down. There's just a lot of churn/NIHS.
(I didn't seem to be getting downvoted, but I also want to be clear that I'm not putting the community down)
END EDIT
I think it's a cool excercise. Why would I ever want to use this in the way it is saying it should be used? Why would I ever use this period?
However, in the web world, ES6 modules have a huge (pardon the pun) advantage that's just being realized. Because of the static nature of ES6 imports, you can eliminate unused code when you package it all together. The newest Webpack versions and Rollup are both taking advantage of this, and it can result in huge savings in filesize.
For example, lodash is a large library with hundreds of methods, and a lot of the time people only use 3 or 4 imports from it. If you use an ES5 version, you'll get all of those functions in a compiled bundle no matter what. If you use an ES6 version with Webpack 2, all the methods you use will end up being the only ones that are actually included in the final code.
Let's say you never import `helperX` and `helperY` from a library, but you import `usefulFunction` which relies on `helperX`. How would webpack know to exclude `helperY` but not `helperX`?
Unless it does some kind of code analysis, or lib is split in "module per function" or similar manner, I can't really see a way to achieve it. And if library is common core plus a bunch of modules per function it exports, how is that different in CommonJS versus ES6 modules?
Am I missing something?
> Rollup statically analyses your code, and your dependencies, and includes the bare minimum in your bundle.
var mod = {};
module.foo = function () { return 42; };
module.bar = function () { return -1; };
module[Math.random() > 0.5 ? "foo" : "bar"]();
Additionally, this method only works with named imports. So for example, you can have two files:
File a.js
export a = 5;
export b = 6;
export c = 7;
export d = 8;
export default {a, b, c, d};
import {a, b} from "./a.js";
console.log(a);
console.log(b);
import a from "./a.js";
Note that it's possible I'm wrong as to the specific optimization method, but largely speaking this is how it should work.
https://github.com/rollup/rollup/blob/6fc8631ba1aad718885151...
This NIHS seems more the result of ADHD and impatience, sprinkled with a little enlightened precociousness.
You can see it in the satire surrounding computer programming. In the 90s, Dilbert had insight into the causes of NIHS. These days, Silicon Valley hits closer to the target.
(PS: the creator of this package looks to be getting a lot of flak here. Kudos to them for doing it, but they would benefit from understanding why it's such a shallow "replacement" for Cygwin.)
I agree. Like I said, I think it's a cool excercise. I'm all for doing things just to learn and explore. Which is what I'd classify this as. The things you learn doing stuff like this will likely help you in the future in some tangential way. No one should be excoriated for practicing their craft, especially in a case like this where it's a nifty project.
The problem is when you put down other projects or even OSes in this case that, frankly, are immensely more powerful and useful than what you've put forth. I think lacking the 'without the suck', it probably wouldn't have gotten much attention, especially the negative.
You didn't have broadband, you couldn't really share entire libraries of code freely, and package managers were not around at the very beginning. In 1995/96 you certainly had message boards and forums, but there wasn't a StackOverflow/whatever like community that made it easy to find an answer to your problem AND/OR a lib to help solve it.
You needed to know how to write a linked list, because you had to write your own. Well maybe not, but you had to write a lot of your own things that just don't need to be written today. God I remember my StringUtils jar from when I was a teenager. That damn thing followed me through college, and I gave the source to friends who then iterated on it (or not). But I didn't have a GitHub to post it to and an NPM to announce, and make it available to anyone looking for a new StringUtils library. It was just mine, free to anyone who asked, but no one knew to ask.
Now your ideas and code and spread to millions of people instantly AND they can find it, which is letting people start from an idea and iterate on it rapidly. Everybody is announcing their StringUtil as the next best thing since taking vowels out of words became cool.
So, I guess yeah- we did thrash 20 years ago, but we thrashed silently and alone.
More often than not, if I'm working on a windows machine, I'm not programming in C. I'm probably working on something in C# or Java. I don't care about the C API. I just want the terminal to work in more or less the same way it does on linux/osx.
Cash seems to be right up my alley, and I'm excited for this.
I personally haven't had a windows machine without git in years. Distribution for windows includes somewhat proper shell with unix utils. I can use the same .bashrc I have on my linux machines, write commit messages using vim, have colors in the terminal, etc...
It's kinda cool I guess (I don't use Windows), I just don't get why the tagline is "Cross-platform Linux without the suck". Apart from the phrase "cross-platform Linux" not making sense in this context, I don't see why other implementations inherently "suck".
Is the "way forward" recombining and rehashing the terrible technologies everyone is familiar with already, indefinitely?
What do you mean by "UNIX shells suck"? I'm fairly sure most people would consider UNIX shells to be one of the bigger revolutions in computing.
But as soon as you want to do anything moderately interesting, it's better to grab a real programming language, because UNIX shells suck. Escaping failures, whitespace handling, almost-real-arrays, fail-and-continue by default, and many many other reasons you'll fail by accident one day is why they absolutely suck for almost anything past basics.
In a shell, any bareword is either one of a few built-ins or an external program. Using a bareword which doesn't correspond to anything isn't a syntax error, because the parser can't know what programs you have installed from moment to moment. That is the biggest difference between a shell CLI and a programming language REPL.
This is, in my opinion, the beauty of UNIX shells. They're simply enough to use like DOS but powerful enough to use like a REPL. It's not perfect; far from it. But few other tools support such a wide demographic.
The shell languages are not much better.
There are some useful concepts, but in general Unix shells are used despite their user unfriendliness.
The shells of the various Lisp Machines were quite different. The Symbolics shell, later called Dynamic Lisp Listener was quite nice on the GUI side and the management of commands, completions, defaults, interactive help, etc..
See for example: https://www.youtube.com/watch?v=o4-YnLpLgtk
The interactive help of that Lisp Machine OS is quite a step up from what any typical shell offers. Though it was mainly developed for single user machines with powerful GUIs.
The problems of that approach: it wasn't very sophisticated on the text terminal - actually it was quite bad. The whole UI mostly assumed a GUI. For development one needed an extended Common Lisp (or Zetalisp), which is a bit too complex for many users.
See also a video I made long ago about the user interface of Symbolics Genera, the operating system of the Symbolics Lisp Machine line of personal workstations.
They're not that bad (GNU coreutils is generally pretty good) and you usually remember the edge cases fairly quickly (eg `dd`). Sadly you get inconsistencies in all coding frameworks, whether it's semantic, function names in the core libraries or whatever.
UNIX shells do have a lot of hidden gotchya's which can make life "interesting" (read: "oh fuck oh fuck oh fuck" lol). But the power of being able to use reusable blocks of any language through pipelining (and to a lesser extent, exit codes) is genius. It means one can mix and match LISP, Java, Perl, Python, C++, Go all inside one shell script.
I do understand the hate towards UNIX shells. There are a lot of faults and a lot of times I I'd be halfway through writing a Bash script and then be wondering if I should have just written it in Perl or Go instead. But no tool is perfect and pragmatically I've found shells to be far more productive than anything I've ever attempted to replace it with. Which is the real crux of why we use these tools. But like anything in IT, this is just my personal preference. Your mileage may vary.
Generally this is all awful and low-level. Just see the UI interface difference between 'dd' and 'Copy File' on a Lisp Machine. The UI is worlds away.
> UNIX shells do have a lot of hidden gotchya's which can make life "interesting" (read: "oh fuck oh fuck oh fuck" lol). But the power of being able to use reusable blocks of any language through pipelining (and to a lesser extent, exit codes) is genius. It means one can mix and match LISP, Java, Perl, Python, C++, Go all inside one shell script.
You can do that on a Lisp Machine, too. With the difference that no pipelining of text is necessary. Just reuse the objects. The data is all fully object-oriented and self identifying.
Note that I'm using Unix and other OS variants since the 80s and I'm fully aware of command line UIs from VMS, SUN OS, Cisco, IBM AIX, OSX, GNU, Plan9, Oberon OS, and various others.
> I do understand the hate towards UNIX shells.
It's not hate. Most Unix shells are just fully dumb. Many people like to use primitive text based UIs with lots of corner cases, which makes them look intelligent for remembering obscure commands, obscure options without real UI help.
Take cp.
The shell does not know the various options the command takes. The shell does not know what the types and the syntax of the options is. The shell does not know which options can be combined and which not. It can't prompt for shell options. It can't check the command syntax before calling it. It can't provide any help when the syntax is wrong. It can't deal with errors during command execution. There is no in-context help. It can't reusing prior commands other that just editing them on a textual base. The output of the command is just text and not structured data. There are really really zillions of problems.
There have been attempts to address this putting different user interfaces on top. For example IBM provided an extensive menu based administration tool for AIX.
> But no tool is perfect and pragmatically I've found shells to be far more productive than anything I've ever attempted to replace it with. Which is the real crux of why we use these tools. But like anything in IT, this is just my personal preference. Your mileage may vary.
Many people have found shell productive. That's why there is a zillion of different shells. You can even use Lisp-based shells like scsh and esh (http://www.serpentine.com/blog/2007/02/16/esh-the-fabulous-f...) or GNU Emacs.
But for most part all these attempts stay in that little box and don't escape the general problems.
> Generally this is all awful and low-level. Just see the UI interface difference between 'dd' and 'Copy File' on a Lisp Machine. The UI is worlds away.
Well yeah, I did already example `dd` as an inconsistency. :)
> You can do that on a Lisp Machine, too. With the difference that no pipelining of text is necessary. Just reuse the objects. The data is all fully object-oriented and self identifying.
Indeed you can. As you can also with DOS, Powershell and so forth. I wasn't suggesting that UNIX shells were unique (though I can see how it might read that way), but that it was UNIX shells which pioneered that concept. For all their faults and the technology that might have superseded it: the idea of pipelining reusable blocks of code was a genius idea for its era.
Powershell also supports passing objects like Lisp does. Personally I prefer the dumb approach; however in all other aspects of programming I do prefer strongly typed languages. This is just personal preference.
> The shell does not know the various options the command takes. The shell does not know what the types and the syntax of the options is. The shell does not know which options can be combined and which not. It can't prompt for shell options. It can't check the command syntax before calling it. It can't provide any help when the syntax is wrong. It can't deal with errors during command execution. There is no in-context help. It can't reusing prior commands other that just editing them on a textual base. The output of the command is just text and not structured data. There are really really zillions of problems.
This part isn't accurate. The original Bourne shell cannot but bash, zsh and fish can all do all of the above. Albeit sometimes (particularly with Bash) you need to install additional helper routines that aren't always shipped / configured with the default package. I believe csh also supports most if not all of the above too.
I'm sure lisp does it better, but I was never arguing that UNIX shells are better Lisp to begin with. Just that I believe Bash et al to have a lower barrier of entry than Lisp. I think on that specific point we might have to agree to disagree - but I don't see many non-programmers using Lisp and this is why I think Bash has a lower barrier of entry (to go back to my original point).
I can completely understand and relate to why you enjoy working inside a Lisp REPL shell though.
Lisp does nothing. It is a programming language.
> Just that I believe Bash et al to have a lower barrier of entry than Lisp
I doubt that, given how horrible Bash as a language and as a shell is. There is no reason why there can't be more sane command systems and better shell languages.
See the zsh documentation on completion:
http://zsh.sourceforge.net/Guide/zshguide06.html
This is all totally over the head of the average user.
_perforce_revisions() {
local rline match mbegin mend pfx
local -a rl
pfx=${${(Q)PREFIX}%%\#*}
compset -P '*\#'
# Numerical revision numbers, possibly with text.
if [[ -z $PREFIX || $PREFIX = <-> ]]; then
# always allowed (same as none)
rl=($rl 0)
_call_program filelog p4 filelog \$pfx 2>/dev/null |
while read rline; do
if [[ $rline = (#b)'... #'(<->)*\'(*)\' ]]; then
rl=($l "${match[1]}:${match[2]}")
fi
done
fi
# Non-numerical (special) revision names.
if [[ -z $PREFIX || $PREFIX != <-> ]]; then
rl=($rl 'head:head revision' 'none:empty revision'
'have:current synced revision')
fi
_describe -t revisions 'revision' rl
}
> but I was never arguing that UNIX shells are better Lisp to begin with
I was not talking about Lisp. I was talking about software. One could write much better command shells in C than what Unix shells offers. I understand that certain people find Unix like shells attractive. From a general user interface perspective they are horrible.
In both cases, for small uses, you can stretch your shell or REPL or programming language over to the other case, but for long term use it's just not practical. After all, in theory many languages have REPLs that could have long since replaced shell, but only a handful of very dedicated people actually replace their shell with a REPL for some language. I've tried, and I can't do it, even with shell support libraries in the relevant language.
The suckiness of UNIX shells is mitigated if you view them as "the tool optimized for interactive usage". And once you let them be that, they really aren't so bad. It's only if you try to force them to be programming languages, especially past a couple dozen lines or when they try to do something other than just run lots of commands pretty much blindly, that they become really bad.
Also complicated quoting is a pain, but as a percentage of the number of commands I run in my shell, they aren't that large, they just loom large in my memory. Run history and really look at what you're doing day in, day out with your shell. Unless you've got a really crazy usecase, there's a lot of things like "ls" followed by a "cd dir" and such. It's part of why I can't get into REPL replacement; replacing "cd tmp" with even "cd('tmp')" is, percentage-wise, a huge increase in keystrokes. (And don't forget, the space is much cheaper than left paren; the space bar is huge and I basically have an entire digit dedicated to it, whereas left paren is two not-on-home keys at the same time.)
I had been juggling the idea of using a modified REPL for shell work, but I haven't written much code. However, someone has written such a shell, namely Avesh[0]. Check out the Reddit comments as well [1].
I haven't used it yet, but the examples seem to fit what I would want in a Common Lisp shell: * Simple sh-like commands with flags (e.g. ls -o, grep -i) * Handling commands and piping as native CL functions (e.g. | (| ls (grep c))(grep r) ).
[0] = https://gitlab.com/ralt/avesh [1] = https://www.reddit.com/r/lisp/comments/48r70b/avesh_the_supe...
Everything's an object. Instead of passing lines of text, you're passing objects, with properties, types, etc. This becomes important because of
Library files are first class citizens. Because the shell is string based, you have to wrap everything up in functions, creating helpers in order to use any sort of function. Powershell makes this trivial, such that you can use functions you wrote elsewhere in your scripts as if they were builtins. Hell, it's trivial to write a webserver that forks multiple processes in powershell.
Yes, the unix shell was revolutionary when it was released, but it's been hamstrung by tradition. The shell is long overdue for letting in more interesting abilties.
Hybrids like ipython shell are quite interesting for personal use though.
I get why Powershell is great for sysadmins (and especially on Windows, where so many system-level utils weren't built with the idea that you'd pipe their output in mind), but for me, it's a step back.
Having said that, I can't vouch for your scripts, but my 10 line shell scripts are unlikely to handle 'edge' cases such as spaces, quotes and backticks in file names.
Also, "incomprehensible to anyone who doesn't know .net" may be true, but your average shell script that builds on tools such as sort, grep, ls, cut, cat, sed, awk, etc. isn't that particularly comprehensible to people who don't know UNIX, either.
Finally, I think it is likely that people who grow up on Powershell feel differently about this.
Absolutely. The point is, "you can use familiar .NET functions and everything is an object" is not something that makes the whole thing easier if you're not familiar with .NET and don't want objects in your shell scripts :-).
Powershell and Unix shell have vastly different descents, and it shows. It's more or less a historical accident that we're conflating them nowadays.
Like everything on the Internet, lack of success with a technology should be taken to mean a failure of that technology as much as a failure of the programmer attempting to use it. It may well have been a problem above the application layer.
If I'm spending twice the time typing on average, even more advanced features won't win me over when I get to use them occasionally (while, again, typing way more on average). It should be saving you time, in the end.
Things are better now with builtin tests, but god help you if you need to use /bin/[ and you end up with idioms like 'if ["x$foo" = "x$bar"]' so that bad things don't happen if $foo starts with a dash, or either variable is empty.
It gets even more fun when you look at the amount of cruft around terminal emulators themselves, and the support for legacy terminals in the command line. Why do modern versions of OSX still ship with /usr/lib/libtermcap.dylib?
Don't get me wrong -- UNIX shells are amazingly powerful tools, but they're just stuck in the 70s and we can do much better today.
I feel like the interface for these commands is exactly what you'd want from a collection of executables that do only one thing.
As for shell scripting, you don't have to use bash/sh/whatever. You could write your script or executable in any language that will let you call system commands, so it's really a non-issue if you absolutely can't stand the syntax. I write plenty of bash scripts and I do feel your pain, they're not usually very pleasant if you have to go beyond trivial use cases- but it's also (generally) not impossible.
Hadn't thought of chaining that with a csv-to-json workflow, that's a neat idea -- but if that's the easiest way out, that's clearly a symptom of the problem I'm describing, isn't it?
One of the things I've been trying to figure out for a while is a way to build a trivial adapter for the csv-to-json | jq pipelines I've been building, something sufficiently easy to use that I could bake it into a |-equivalent extension to something like zsh.
The security ceremony required around PS is also an obstacle to getting started. And of course Microsoft have missed the opportunity to fix the path separator.
I admit that Bourne shell is not great for anything over a few lines, and find it better to drop into perl or python where filename-separator issues are mostly eliminated.
Sounds like you are looking for Tee-Object[0].
"Saves command output in a file or variable and also sends it down the pipeline."
And of course Microsoft have missed the opportunity to fix the path separator.
Windows supports both path separators in pretty much all cases (open files, changing directories, etc)
PS C:\> d:
PS D:\> cd /projects/node_modules/immutable/node_modules/uglify-js/node_modules
PS D:\projects\node_modules\immutable\node_modules\uglify-js\node_modules>
I guess this depends on what you want out of the system. Some options are:
"format-table -auto" which provides a nicely formatted table view of the data
"format-list" which provides each property on a separate line with a blank line between objects
"format-custom" which allows you to create your own views of objects (with a very readable default), which has ultimate flexibility
"converto-csv" which output the object as a CSV (or with custom delimiters such as TSV)
"converto-html" which outputs the object as an html table (with somewhat customizable markup)
"convertto-xml" similar to the above
But really, if it's possible, then it makes sense to use the object pipeline until you're done (so that different parts of the pipeline can use different parts of the object). Then spitting out CSV, TSV, XML, etc should get interop with basically every other tool that's going to process text in some way.
[0] - https://technet.microsoft.com/en-us/library/hh849937.aspx
The output of tee-object can't easily be read back in to a new pipeline. I suppose we're supposed to stash it in a variable which will be good enough for 99% of cases.
There's still a lot of room for improvement.
1. Win+R -> cmd enter is a lot faster than Win+R -> powershell enter
2. cmd starts faster than powershell
3. I seldom require all the power from powershell
npm install cash-global -gAs a web developer, I know JavaScript very well. I have already encountered thousands of its idiosyncrasies and I am very good at making it work for me.
Therefore any tool written in JavaScript is automatically easy for me to understand and modify. If that tool could have been more elegantly written in another language, that's awesome for people who know that other language, but it's irrelevant to me. Even if I kind of know my way around the other language, I'm never going to be as effective with it as the language I use every day. And a lot of people use JavaScript every day. This is the reason why people feel the need to solve problems in JavaScript.
I understand people feel the need to solve problems in JS because they're scared or inconvenienced by learning a new programming language.
But why not learn a new language? Having a working knowledge of C won't kill you.
edit: could you explain how JS is the most widely used language in the world?
I do have some knowledge of C, and I could learn more. But unless I change career, I'll always be faster and more effective in JavaScript because I think in it all day, so I'll almost always choose JS over C. I'm not disparaging anyone who uses C, just answering @faaef's question about why people "feel the need" to make everything in JavaScript – because they can, they happen to know JS very well, and because there's automatically a huge potential contributor base.
> could you explain how JS is the most widely used language in the world?
I don't know, but I assumed it's the most widely used because websites :) Also it's the most popular language on GitHub
Like this one. Like the POSIX API that is actually compatible with other applications.
And by that definition of used, C and Java are pretty strong competitors, seeing as every SIM card and Bluray player and so many more run Java... and then there's Linux and its embedded RTOS cousins, etc., etc., that are written in C or C++. I think embedded devices beat out JS pretty well.
Apart from that, it's great to have a favorite language, but choosing a hammer for every task does not seem like the mark of a great craftsman for me.
Perl/Python/PHP/Ruby: Generally 10x or more slower than V8 Node.js, plus global interpreter lock = automatic no.
C/C++/etc? Fast, but too complex to work with build system / platform compat / 3rd party modules = generally no.
Java? Way too slow startup time; JVM install for users painful = automatic no
Scala/Clojure/etc: Fine languages but same practical problems as JVM languages = automatic no.
Go? Probably nice, but I just don't know it = maybe
Node.js: Fairly fast, easy to debug, fast startup time, easy to use 3rd party code = yes
Rust: safe, C perf, and easy to build and use 3rd party modules via Cargo = yes
As long as IO is your bottleneck the GIL shouldn't be much of a problem. And Python (and I'm sure Ruby, too) has excellent async support.
Not saying Node.js isn't useful, but I wouldn't automatically say no to Python or Ruby.
Making computing worse, one fork at a time.
This is presented as Unix Without the Suck, when in fact it's almost 100% suck.
If it were described as just a simple humble Node.js shell without the puffery and unnecessary bash at Cygwin, I doubt you'd see the same vitriol.
(Personally, I could use a little less of this trend in computing for everything to be awesome and everyone to be rock star ninja guru superstars. For reasons like the responses here show.)
Sure, Node.js has a big community with lots of (mostly linux-only) packages, but you get a mediocre language and pretty crap cross-platform support. This doesn't matter as a web development language, but seriously gets in the way when you're developing a program designed to do literally everything.
A more useful project might be coreutils in rust, which several projects are already doing.
P.S. I'm constantly amazed by the number of non-JS developers that pile onto JS-related HN threads to tell us how much JS sucks. I'd like there to be less of it, it's getting a bit old. It seems like any amount of familiarity with the web, or the fact that someone once wrote a JS function sometime makes people feel entitled to weigh in on modern JS development. Look after your own communities and let us plough our own furrow ... we think lots of small modules is an interesting approach, and we'd like to see where we can get with it.
All the bulk is just the utilities that people want to provide once they've got that layer (i.e. CoreUtils).
I can see the usefulness in a JS shell mainly because JS is a popular scripting language.
Running this on Linux will be difficult because node runs with a very limited set of permissions.
- short-term : POSIX compliant
- long-term : runs *nix binary / programs
p/s: love anything CLI, so I may be biased
OR
maybe I didn't see their long-term goal?