An Introduction to Unix (2014)
oliverelliott.org
oliverelliott.org
"Under the hood Windows, unlike Mac OS, is DOS- rather than unix-based"
Last DOS based Windows was released like 15 years ago. At that time Macs had their own weird OS too.
"A general point about unix commands: they're often robust."
Unix commands, robust? You gotta be kidding me, they are great for quick ad-hoc hacks, but robust they are not. Just as an example here is how the common task of handling can fail: http://www.dwheeler.com/essays/filenames-in-shell.html
"For example, with ls you can use an arbitrary number of arguments and it obeys the regular expression (more about these later) convention that an asterisk matches anything"
How this is an example of robustness? Besides globbing is not regular expressions. In regexp you'd use .* to match anything, not a bare *
> However, a file in Microsoft Word's proprietary and unknown formatting is utterly unusable.
While I'm no fan of Word, it is very much arguable that Word file format not anymore proprietary or unknown.
> I used to use Smultron until I found, unforgivably, that the spacing of documents when you looked at them in the editor and on the terminal was different.
Let me guess, tab width difference? Yeah, that is how tabs work and not really a fault in the editor.
For the first time ever I had to deal with xargs and find together last week. Bear in mind I've been using Unix for about 20 years now. Telling one command to null delimit stuff and another to parse the nulls is horrible. Especially seeing as this only came about because one directory happened to have a lot of files in it and the 11 year old script just went snap. I'm perhaps lucky I've never encountered this.
The author needs to take a look at powershell. Surprisingly I wrote a job site scraper in it in about 10 lines of code. Html parsing, data access etc are right there at your finger tips. That and it'll quite happily move 23TiB/1780000 files in a single incantation without wincing once.
Colour me impressed (as an old Unix die hard)
I'm really curious about this. Do you have a very specific workflow that gets around that need or an alternative that would replace find + xargs? Is your job pretty niche? It just seems like I _need_ it a few times a week minimum, and I'm not really sure what would replace it off the top of my head.
I do agree on powershell being quite slick. Sometimes finding what I want is a process, but so was unix in the beginning.
But more seriously, it only looks like the Unix shell. Operators like '|' can have very different semantics, especially when passing objects around and whatnot.
http://www.gnu.org/software/parallel/man.html#DIFFERENCES-BE...
One thing some people might not know is that DOS itself is slightly Unix-based; e.g. the hierarchical file system (directories within directories), command-line piping (though limited in features as compared to Unix), even the MORE command (ditto), etc. This is as opposed to, and an improvement over, CP/M, for example (a popular predecessor OS to DOS, on mainly 8-bit micros), which didn't have a hierarchical file system (IIRC - I've actually used both, CP/M lightly, DOS a lot, ages ago).
http://en.wikipedia.org/wiki/MS-DOS
If you are on Windows, you would gain quite a bit more from learning Powershell, not Unix. Powershell has taken many lessons from Unix, and (dare I say) improved on many things. It's absolutely worth learning, if you are going to work extensively on Windows.
But if you're going to work in *nix and Windows, your bash/cygwin skills/experience will be useful in both locations, and much of your work will be reusable between the two worlds.
Powershell is Windows-only, which is fine if your life is Windows-only or Windows-heavy.
Not that there's anything wrong with knowing and using Powershell.
Here's a nice response to the question of whether PowerShell is ready to replace Cygwin [2], circa 2009
Cheers!
[1] https://ramblingcookiemonster.wordpress.com/2013/12/07/why-p...
The grandparent clearly frames it like this; "if you're stuck with Windows...". He's not saying "If Windows is your choice of operating system...".
Virtually all the scripts I write run without problems in either environment, because it's python. Scripts that other people wrote for Windows run fine in cygwin. There are occasions where I have to account for differences in the OS, but mostly never.
I didn't look back so I don't know what was the main reason but I can say there are some problems.
2424 pip install youtube-dl
2425 which youtube-dl
2426 youtube-dl --help
2427 youtube-dl https://www.youtube.com/watch?v=WS-JcaPFzp4
[youtube] WS-JcaPFzp4: Downloading webpage
[youtube] WS-JcaPFzp4: Extracting video information
[youtube] WS-JcaPFzp4: Downloading DASH manifest
[download] Destination: The Renaissance and the cuckoo clock-WS-JcaPFzp4.mp4
[download] 100% of 7.98MiB in 00:02
2428 python --version
2429 which python
2430 history |tail
This worked fine for me. It installed, help was available, I downloaded the video and watched it.I suspect one of two things, although it could be something else. You might be using the windows installation of python, rather than the cygwin installation. In most cases that works fine, but cygwin's python is compiled to expect a cygwin/unix-like environment, and windows python is compiled to expect windows. Sometimes weirdness ensues, and it's better to use cygwin's python in cygwin.
My other suspicion is that the installation of youtube-dl didn't happen correctly. Notice above that I used pip, which is offered in the installation instructions. If you've installed pip that's usually your best/first option. Their first instructions involve downloading the gz (as you did). I did that with wget in python, then noticed the installation is an extraction and a chmod; I'm usually more comfortable trying pip in that circmstance.
I've placed my email in my profile if you want to chat about this. I'll leave it there until I remember to remove it.
Edit: you can run the python shell, and it will summarize python's construction:
$ python
Python 2.7.8 (default, Jul 28 2014, 01:34:03)
[GCC 4.8.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
From a windows cmd.exe shell: python
Python 2.7.6 |Anaconda 2.1.0 (64-bit)| (default, Nov 11 2013, 10:49:15) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> which -a python
I discovered this when I ran pip and ipython in cygwin, and they "worked," i.e. something ran, but not right.I put this at the end of my .bashrc to remove windows python from cygwin's path:
export PATH=$(echo $PATH |sed -e "s/:[^:]*Anaconda[^:]*:/:/g" |sed -e "s/:[^:]*Anaconda[^:]*:/:/g")
There were two anaconda directories in the path, and /g should have taken care of both, but my RE-fu is apparently lacking, so brute force.Cygwin has pitfalls.
1. Get and read The UNIX Programming Environment by Kernighan and Pike.
https://www.google.co.in/search?q=the+unix+programming+envir...
A classic. Outdated [1] some, for modern Linuxes and BSDs, so:
2. Supplement it by reading Linux / BSD man pages and get a book or two on one of those OSes, or other Unix variant you are using, such as Mac OS X.
[1] But you are unlikely to find another book that better conveys the Zen of UNIX, so to speak. The authors were early UNIX users and developers, and Rob Pike (now at Google) is one of the creators of the Go language.
Well, that is to be expected given the fragmentation in the UNIX world.
One never knows how the specific UNIX is going to behave, which command line arguments are available besides the common ones and what sort of POSIX undefined behavior exist.
You will evolve your own style of command line usage from what you read and what you are able to remember. Even very skilled people will do things differently.
If you want a classic though, check out Unix Power Tools at O'Reilly
EDIT: changed ascii asterisk in front of all instances of nix to u+2217, the plain asterisk translates to large bodies of italicized text on HN. Which I knew.
Yes, that's very good. If I was to compare it with my recommendation in this thread, of The UNIX Programming Environment (UPE), I'd say that the Power Tools book is like a cookbook, while the UPE book is like a tutorial (but a power tutorial :)
http://jugad2.blogspot.in/2014/09/my-ibm-developerworks-arti...
And this post shows a practical use of that utility:
Print selected text pages to PDF with Python, selpg and xtopdf on Linux:
http://jugad2.blogspot.in/2014/10/print-selected-text-pages-...
[1] In C, but the guidelines shown are also applicable to writing Unix commands in other languages like Python and Ruby.
http://linuxcommand.org/tlcl.php
I consistently recommend The Linux Command Line by William Schotts. It's available in print through New Starch, or on the web. It's very clearly written, and practical.
http://shop.oreilly.com/product/9781593273897.do
2nd review from the top (currently), reviewer id is vasudevram.
Bravo!
>(2) it's the most natural port of entry into all other programming languages;
Unix is not a language, but an OS.
I agree with the OP that conceptually, the command-line is the best place to start learning programming...the idea that a tool should be kept simple, but that it should be easy to connect things together, and that it's up to the human operator, not the computer, to do this work...this is very powerful, and something I wish I had learned early on as a programmer. The way that Unix tries to think of everything as text is also a powerful concept, because for most beginners, they think of file formats as being "magical" and immutable, they don't have the concept that at the core of it, everything is just binary (everything being "just text" is a nice middelground from OS magic and trying to read binary)
However, the beauty of this philosophy is hard for non-programmers to appreciate or take advantage of. Moreover, things that I love as a programmer, such as silence-by-default and sparse error messages...are massive stumbling blocks for beginners. While I've done enough programming to appreciate silence, it's another thing to be a beginner who has never edited code in a text-editor to deal with "silence"...perhaps the biggest obstacle is the high difficulty in debugging. I can generally pinpoint any error in seconds...but this is only after years of general programming. The way that Bash continues on when an undeclared variable is referenced (via a typo) has led to hours of frustration for my students.
I think next year when I teach the class, if I do Unix again, I'm going to commit the first two weeks to muscle memory and flashcard-like memorization...it's too difficult to get the higher-level concepts of programming while you're stumbling around with recognizing variables. On the other hand, I think I may not try to teach Unix again, and will just stick to Python. One of my goals of teaching Unix was to avoid complexity and especially, things like OOP...I think Python can be carefully taught to fit that route...though I'll miss the ability in Bash to do such real-world tasks as curl, parse, and analyze a webpage in just two lines.
Edit: There was also a practical reason for doing things in Unix...all my students have different kind of laptops and OSes...and Stanford's shared computing runs on Ubuntu. So they did all their assignments on the shared computing cluster. That led to another issue of not being able to easily edit script files, in the way that you can do with Sublime Text on your own OS.
I counter: 'Please enable HTML!'
In all seriousness, it's text, images, links. There are no user accounts, no dynamically updated database of 'favourited' sections, no per-paragraph instant chat, the page is remarkably atrocity-free!
Hence, this page seems very much un-Unix-y: Solve problems at the lowest level of complexity, what?
You can get around this by opening up your web inspector, and disabling the `hidden: true` rule on the top `container` element.
Nothing on the page (or in related linked pages on his site) seems to require JavaScript.
View/Page Style/No Style
Then reload.
I think the No Style menu item should have a middle finger for an icon.
Sort of ironic for a site that teaches and recommends Unix.
PS: Only slightly related, but I'm currently working on a SMTP replacement that focuses on speed, privacy, and ease of use. I believe that if more people would do this, at some point some new technologies will stick and we'd get rid of the (bad) 80s-90s stuff.
But why should we accept this? Why does it justify pages like this one, that don't need javascript but deliberately fail if you don't enable it?
Some companies won't target your browser if you have less than 5% market share. Even 10% market share. So imagine when I find out you are less than 2% market share (no js users).
For my web dev company, we have about 10 clients. None of the visitors have ever had javascript turned off that I'm aware of over the last several years.
That doesn't mean the days of "designing without javascript" are over, though. HTML and CSS work more or less the way they always have.
If I don't have to go to your site, and it wants to force me to use JS when it doesn't need JS (for example, when it's basically a bunch of static text), then I'm just not going to visit your site. If that's ok with you, it's ok with me.