What's the one Linux command you wish you knew years ago?
reddit.com
reddit.com
disown - bash built in.
You know how you always start that mega long rsync in an ssh session from your laptop and then realize you have to go offline halfway through? You forgot to start it in screen or nohup, didn't you? You can pause it (ctrl+z), background it (bg), and then disown it so it is protected from SIGHUP when you quit your ssh session. Bash (and, to be fair, zsh and others have copied much of this) has some wonderful job control features most people are completely oblivious of - even veterans of 20 years.
There's also retty: http://pasky.or.cz/dev/retty/ but again it's not a complete solution.
GNU screen, which I love and use to obscene extents daily, has an exceptionally hairy codebase. That said, I've got to get my tmux on and learn how to do screen equivalents with it....
http://www.techrepublic.com/blog/opensource/is-tmux-the-gnu-...
Since it is installed for the most part everywhere I have no desire to switch to tmux.
If there are problems found with screen (and it's a widely used admin tool that may be left running on remote servers, hence a high-value attack vector), it's going to be harder to find and fix the problem.
If there are features to be added, they'll be harder to add to screen than tmux. Which means that with time, tmux stands a higher chance of becoming more featureful.
Both argue in favor of tmux in the long run, though there's no immediate need to ditch screen. As you note, screen's very widely available, which is a feature. Then again, so was telnet 10 years ago.
Screen lets each part of a split screen select a different existing window. Tmux makes both parts of a split part of a single window, so you can either have two things going in one split, or have something else going on in another window, but if you suddenly decide you want two existing windows side by side, you can't do it.
OK, I looked back and it's fixed now: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621804. 6 months is a long time to wait. (Not that I'm criticizing Debian. It would never be fixed if it wasn't for the distributions patching it.)
I currently use dvtm to split terminals and connect to multiple screen sessions in a single window. The downside of dvtm is it takes the dwm[3] approach of using #defines rather than configuration files.
1. http://www.brain-dump.org/projects/dvtm/ 2. http://dtach.sourceforge.net/ 3. http://dwm.suckless.org/
I can't remember the exact details, but that should be enough to set you up for some googling.
http://www.commandlinefu.com/commands/browse/sort-by-votes
Most of my time savers that I've never thought of were found there.
Love this one.
python3 -m http.server
[ward@hathor ~]$ ls -l /usr/bin/python
lrwxrwxrwx 1 root root 7 Sep 5 00:45 /usr/bin/python -> python3
Arch default mv /path/to/some/file.{jpg,png}
Has been a huge timesaver. It's the same as mv /path/to/some/file.jpg /path/to/some/file.png
But saves you the typing/copy pasting the path for the second argument. Can also be used with cp etc. rename jpg png /path/to/some/file.jpg rename 's/jpg/png/g' /path/to/some/file.jpgI prefer the regex one, of course, especially since it takes proper Perl regexes :)
You have to be careful of this though and probably avoid using this command in shell scripts.
edit: You can just stick Debian's rename at $HOME/bin/rename.pl and call it a day; this command is useful in the terminal since it's less typing than a for loop.
http://www.gnu.org/software/bash/manual/bashref.html#Brace-E...
And it's even better in zsh.
http://zsh.sourceforge.net/Doc/Release/Expansion.html#Brace-...
> ls *.{png,jpg,bmp,gif}
But this gives error if there is no jpg files. In bash, you can activate the extglob mode with "shopt -s extglob" and do this : > ls *.@(png|jpg|bmp|gif)
http://www.faqs.org/docs/bashman/bashref_35.html $ diff -u <(cd a && find . | sort) <(cd b && find . | sort)
Internally, bash creates a pipe and passes /dev/fd/XXX to the main command. $ ls a b
a:
1 2 3 4
b:
1 2 3
# Yours:
$ diff -u <(cd a && find . | sort) <(cd b && find . | sort)
--- /dev/fd/63 2011-11-20 02:50:21.919072726 -0700
+++ /dev/fd/62 2011-11-20 02:50:21.919072726 -0700
@@ -2,4 +2,3 @@
./1
./2
./3
-./4
# Mine:
$ diff -u a b
diff -u a/1 b/1
--- a/1 2011-11-20 02:43:57.073130007 -0700
+++ b/1 2011-11-20 02:45:14.535912768 -0700
@@ -1 +1 @@
-1
+11
Only in a: 4
Maybe I'm missing something?screen, pdsh,
perl -pi -e 's/find/replace/g' *.txt
redis-cli's new ability to have piped in data be the last argument of a command.
vim -o file1.txt file2.txt file3.txt (opening multiple files with vim)
Using vim instead of less as a pager: cat file.txt | vim -
A hack for in-place editing, for old school sed (no -i):
sedi(){ case $# in 0) echo usage: sedi cmds file;; 2) sed -an ' '"$1"'; H; $!d; g; w '"$2"'' $2;; esac; }
vimpager. Set vim as your pager. I love reading man pages in syntax-colored vim glory.
'most' will colorize bash output as well (it's strictly a pager, in the pg / more / less / most continuum).
Unless you only use GPL software, there's no reason to not use tmux these days. Well, maybe if you're on HP-UX or something.
However, it's not true that screen has no advantages over tmux. Each program has its own strengths and weaknesses. For instance, screen is scriptable via its Lua bindings, tmux is not. Screen has zmodem support built in, tmux does not. There are probably many other examples, since screen has a bazillion features which have been developed over decades, while tmux is relatively new (and its developers don't seem to care to duplicate every one of screen's features).
It's certainly true that screen does have tons of small old features... but I sort of swept that in with "unless you're using HP-UX or something." zmodem support isn't exactly relevant to the overwhelming majority of programmers, but yes, I admit these things are still relevant to a few niches.
I'm not sure if tmux is scriptable to the same extent that screen is via screen's Lua bindings, as the latter probably exposes many of screen's internals, while scriptability of tmux is limited to what you could do with its command line arguments. I haven't researched the matter, though, so I may well be wrong.
I often have to fix servers that are not from own group, so all my knowledge is honed around tools that i can find anywhere.
# put this in your .screenrc
escape ^Oo escape ^\\It works because I have my shell configured to use vi mode for command line editing.
It, along with other readline functionality, shell functions, substitutions, expansions, scripts, and the bazillion utilities are what make Linux (and Unix) shell so much more than just "it's like DOS, right?".
Yeah, kind of, in the same way that ... a cross between a Prius, a Mack Truck, an Lamborghini, an F-16 fighter, a helicopter, and a freight train are like a pushcart. It's an interface that helps you manage your computers, the things on them, and the things they're connected to. It's also a hugely efficient and effective way to process information and issue commands and controls in a useful way.
Step 1: Think "I wish I could do X."
Step 2: Google/duckduckgo "How to do X in bash"
Step 3: Delight that a simple solution already exists.
This method is successful in 95% of all cases.
Also, `` and $(). Find combined with one of these is great. One of my favorite one-liners (I even figured it out myself :)) is:
cat `find . --name *.java` | wc -lfind . -name *.java | xargs cat | wc -l
find . -name *.java -print0 | xargs -0 cat | wc -lThe command line can be a large as available memory (for a single processes). (Although the ratio between command line length and allocated memory is not obvious since it depends on the number of words in the command line.)
find . -name '*.java' -print0 | xargs -0 cat | wc -l --max-args=max-args
-n max-args
Use at most max-args arguments per command
line. Fewer than max-args arguments will be used
if the size (see the -s option) is exceeded,
unless the -x option is given, in which case xargs
will exit.The xargs utility shall limit the command line length such that when the command line is invoked, the combined argument and environment lists (see the exec family of functions in the System Interfaces volume of POSIX.1-2008) shall not exceed {ARG_MAX}-2048 bytes
so you should never run into the ARG_MAX limit.
find . -iname '*.java' -exec cat {} + |wc -l
But if you do that, why not just: find . -iname '*.java' -exec wc -l {} +
Which will also print out output of wc -l for each files along with total lines.E.g.: I like dumping temporary files with timestamps:
some-command-to-generate-log > /tmp/log-$( date +%Y%m%d-%H%M%S )
Now, if I want to wrap that in a larger loop -- say, iterating over a number of files or parameters:
command $( expansion one $( expansion 2 ))
Works
With backticks you'd have to do escapes:
command `expansion 1 \`expansion 2\` `
... which gets tedious.
because it requires less typing. The ISO 8601 standard format.
-Is, -Im, -Ih, and -Id allow you to change the displayed resolution.
Bizarrely, this doesn't seem to be documented in my version of date (coreutils 8.10).
That said, I can't find "-I" documented anywhere.
What I like about the timestamp I use is that it's semantic but sorts lexically as well. Though yours does as well. Hrm.
The {} means "replace with filename" and the semicolon (escaped so your shell doesn't eat it). This will run wc -l <filename> for every file that matches.
The -regex flag lets you do the same thing as -name with regular expressions.
The problem with this is we run n different wc's, as opposed to just one with all the parameters. We can use xargs to fix this: find . -name .java | xargs wc -l
This even gives you a total!
The really* simple solution (which is probably best) is to just use shell globbing: wc -l *.java
While find and xargs are definitely useful, the easy solution here works better
-exec utility [argument ...] {} +
Same as -exec, except that ``{}'' is replaced with as many path-
names as possible for each invocation of utility. This behaviour
is similar to that of xargs(1).
The problem with just wc -l *.java is that it don't recursive directory. wc -l **/*.java wc -l $(find . -name '*.java') find . -name \*.java -exec cat {} + | wc -lAlso, find may or may not split the files it finds into multiple invocations.
find . -name \*.java -print0 | xargs -0 cat | wc -l
Edit: Dammit. Others have already posted this exact version further below. :-(
I just use M-x find-dired and friends these days for most things, admittedly.
find . -mmin -5 -print
zsh also has the 'zargs' command, which works something like xargs, except that it gets its arguments from the command line (often in conjunction with extended globbing) rather than from stdin.
You might also be interested in reading about the "useless uses of cat"[2]
cat **/*.java | wc -lIt's not as useful as find.
I could've used this many times to catch cases where the amount of work being performed didn't match my expectations; usually an indication of an incorrect assumption or mistake I had made. Instead I sat there waiting for something way longer than I should've, only because I had a bad feedback loop.
Oh, and "sudo !!". Definitely key, along with "& disown".
Host somewhere
HostName somewhere.example.com
User username
IdentityFile ~/.ssh/somewhere_key.publet you set up hops when you don't have direct access to a server.
e.g. if you don't have direct access to serverB but have direct access to a serverA that has direct access to serverB
put this in your ~/.ssh/config file
host serverA
hostname serverA.domainname.com
host serverB
hostname serverB.domainname.com
ProxyCommand ssh serverA "nc %h %p"
then typing ssh serverB will log you on serverBBut it's also terrifying, because it exposes how many people will post questions before using man.
13 pages of the 80 page manual are devoted to readline and programmable completion. Readline and programmable completion could be removed from the shell, and their functional equivalents moved to a more logical place in the terminal emulator. Plan 9 does this, and it works well, and people seem to like it and the flexibility it gives.
24 pages are devoted to builtin commands, many of which are unnecessary duplicates of regular commands, and several unnecessary altogether.
3 pages are about weird parameter expansions, all of which can be done more intuitively with sed, except for the array one, which can be done with awk.
And there are many smaller things as well, like biffing or arithmetic evaluation, which can be done with dc or whatever. You could try moving history out of the shell and into the terminal emulator as well.
$ man bash | wc -l
5351
$ 9 man rc | wc -l
496
You may want to check out the `rc' shell from Plan 9 (carried over from latest UNICes). Simple, elegant. Available for linux via http://swtch.com/plan9port/
Manpage at http://swtch.com/plan9port/man/man1/rc.htmlThis is how I learned unix shell back around 2000ish. Aside of giving an excellent insight into zsh, it also gives many good hints and notes about unix shells in general.
The guide is in some serious need of a good editor to make it more concise and better organized. It also needs many more simple, practical examples. Unfortunately, it hasn't been updated since 2002, which also makes it a bit obsolete, since zsh development has been very active since then and zsh is now on to a new major version.
Overall, the guide is a nice try, but zsh needs more and better documentation if its not going to overwhelm all but the most dedicated users.
$ help "*"
will print out all shell built ins (list help for them). And if the short description is not enough, go google for each.
Edit: This works rather well:
function cmdfu() {
curl "http://www.commandlinefu.com/commands/matching/$@/$(echo -n $@ | openssl base64)/plaintext" --silent | sed "s/\(^#.*\)/\x1b[32m\1\x1b[0m/g"
}
From http://www.commandlinefu.com/commands/view/3086/search-comma..., it uses the commandlinefu API: http://www.commandlinefu.com/site/apiI'd like to see someone dump all the common man pages into a public wiki so people could flesh out the usage examples and suggest alternative tools.
Most Linux distros have a bug reporting feature which could be used to suggest such examples, and it would be a Good Thing if more were suggested.
Of course, watch out for differences in the utility. OpenBSD (usually) won't have any GNU extensions, for example.
You can simply view the original file (it has minimal markup) in a pager for a single-file view. I generally write an "info" script or function to replace the info viewer with something more useful.
I share your dislike of the info format.
... gives you paginated manpages. I get about 96 pages for bash 4.1.5.
man -Tps bash > /tmp/bash.ps && gv /tmp/bash.ps
... gives you the manpage pretty-printed as postscript output (assumes you've got the gv ghostview viewer installed). Or you can use a PDF converter (e.g.: ps2pdf) and read with your preferred PDF viewer.
This version runs about 70 pages for the same manual.
For me, often asking the question and then still trying to figure it out myself worked well. Often I come to a solution, post it (IRC) and then later some guru responds with "try this instead" or "with that flag it could do some thing better".
The pool of knowledgable people decreases with each question asked, especially if you're not making an effort to at least try to do it yourself.
What happened that it's possible for someone to not know that it exists?
If you search for help with command line utils you are almost guaranteed to get some results that are man pages online, so the person who said that probably has read man pages without realizing it, and without the convenience of the man program.
These days most people just know that if they have an unanswered question, ask the Google.
And that also happened. It used to be virtually impossible to jump in alone, because someone had to create an account for you. It's now possible to start in Unix with no one's help. As long as you can figure out how to burn an iso to CD, you're in. The installation gives you an account, with sudo.
After you install CrunchBang, it pops up a terminal the first time you log in, and allows you to make additional choices that weren't made during install.
I think all distros should pop up a terminal upon first boot and direct you to read "man man."
(or after you start emacs, M-x server-start). This starts up the emacs server, so that it preserves your open files, and you can start new emacs instances immediately with emacsclient. (I actually symlink emacsclient to vi since it loads as fast as vi, which was one of the weakest points of emacs for me.)
Very useful if you are using a remote box (like a vps) as your dev environment.
export ALTERNATE_EDITOR=''
alias e='emacsclient -t'
Now $ e file.txt
will fire up emacsclient to connect to a running emacs daemon. If there is no emacs daemon running it will start one for you and then connect to it.You can
export VISUAL="emacsclient -t"
export EDITOR='emacsclient -t'
for good measure if you like. $ vim -h | grep -i server
--remote <files> Edit <files> in a Vim server if possible
--remote-silent <files> Same, don't complain if there is no server
--remote-wait-silent <files> Same, don't complain if there is no server
--remote-send <keys> Send <keys> to a Vim server and exit
--remote-expr <expr> Evaluate <expr> in a Vim server and print result
--serverlist List available Vim server names and exit
--servername <name> Send to/become the Vim server <name>
$tee piping to a log file with tee, while still seeing normal stdout on screen
This assume Emacs-style bindings for the shell.
alt-h behaves similarly but runs `man <command you were typing>`, so if you have this:
$ rsync -
and need to check the options, hit alt-h and the man page for rsync comes up. When you exit you'll be back at: $ rsync -my ~/bin folder has ~50 different little scripts that handle all sorts of small tasks.
awk -F, '{print $3}' foo | xargs -I{} echo 'command -x -y -z {}'
perl -e 'open F, "<foo"; while(<F>) { my($a,$b,$c,@rest) =split /,/; system("command -x -y -z $c"); }'
I think I could do the split into an array and then take the third element, but I'm just trying to do an elementary example. When you want to do more things to this argument you necessarily have to grow this, and at one point I got to the point where I started having to escape quotes. That's rather difficult to read when you're just trying to do something quick in the command line and you mismatch a pair.
perl -lane 'system("command -x -y -z $F[2]")' foo* Copying files across machines (`cat file| nc -l port` and `nc host port`)
* Remote pasteboard (on OSX w/ pbcopy, `while true; do nc -l port| pbcopy; done`)
Not a life changing, but I wish I know it earlier.
Only if there's a way to make "remote $EDITOR" with Netcat...
If you frequently connect through VPNs to other hosts, often enough the SSH connection times out and just siits there, taking no commands. Hitting ~ and . will kill the session (it's an openssh feature).
Other frequently used tools: awk, sort, uniq, tail, find, grep.. they alone make the shell a really powerful tool
so:
<ret>~? (list possible escapes)
<ret>~. (disconnect session)
If you're dumping binary data through ssh as a pipe, this can sometimes bite you if these sequences appear in your data. ssh -oEscapeChar=none ...
handles that situation. (for the gory details, man ssh_config)But here is mine that I recently learned:
sort -h
compare human readable numbers (e.g., 2K 1G) du -h | sort -hYou can't do sort foo > foo because you'll be left with a blank file.
Although sort(1) handles this for you as you mention, the general problem in the shell of wanting to overwrite your input file with the output file, but not being able to redirect to it for the reason you mention, is solved with sponge. Rather than:
$ grep "foo" bar.txt >baz.txt; mv baz.txt bar.txt
instead:
$ grep "foo" bar.txt | sponge bar.txt
Passes last argument on the command line. In general, the whole paragraph on HISTORY EXPANSION in the bash manual is a must read.
A few ones from the moreutils package:
- vipe : manually edit the data in the middle of a pipe.
- vidir : treat a directory and its files as a text file. Does renames and deletions.
- sponge : when you're doing modification on a file in a pipe, and want the output to be to this file (file will be scrambled without sponge).
It has others, very useful tools, but I won't spoil them all.
!! is short for the previous command, but you can do !-2$ to get the last argument from the 2nd last command, or !cp$ to get the last argument from the most recent cp command. Or !cp:2 to get the 2nd argument to the most recent cp command. !# refers to the current command being entered (in zsh, don't know about bash).
Basically I second Aissen's recommendation to read up on history expansion. Super useful.
socat TCP-LISTEN:80,fork TCP:localhost:3001
awk '!x[$1]++'This can be mimicked by brace expansion: cp /dir/dir/dir/dir/{filename,newfilename}
echo hi > /tmp/blah.txt
ls -l <esc-.>
There's a bunch of complementary commands to go with it too.alt-. works too. In the shell ESC <foo> is the same as alt-<foo>. The difference being that with alt-<foo> you hold alt and press <foo>, then release them both. with ESC <foo> you press escape, release, and then press and release <foo>.
cd -
to change back to whichever directory you were just in. Every once in a while most of us find ourselves facing someone's "how could you not know that?" look.Recently I had this thought: maybe those people who can remember the options actually are autistic (don't know the proper word, that thing many geeks are deemed to be), and I am not. Remembering command line options might be like knowing large prime numbers within seconds.
But I am also interested in memory techniques like the Loki method. Perhaps some good system for memorizing command line arguments could be devised...
ps -A | grep some text in program name
upon getting the pid
kill -9 pid
killall programname
Most of us probably won't be executing killall on any HPUX or IRIX boxes, but it's still good to know the difference ;)
pidof
There's a nice write-up and a function automating attempts to run increasingly strong signals on this site: http://web.archive.org/web/20080610070315/http://sial.org/ho...
(Thank god for the wayback machine. That site was full of good Unix tips and tutorials, but it's gone now.)
872K ./Desktop
78M ./Documents
3.2G ./Downloadsedit: requires simplejson
Saved my work on remote servers too many times to count. :)