Ask HN: What are your favorite terminal programs? | Hacker News Reader

Ask HN: What are your favorite terminal programs?

279 pointsorcs199 comments

The title and why?

Recently discovered CMus and am a big fan of Pandoc and youtube-dl.

This has me wondering what else I'm missing out on.

Written in Rust:

    - fd: A simple, fast and user-friendly alternative to 'find'
    https://github.com/sharkdp/fd

    - ripgrep: ripgrep recursively searches directories for a regex pattern
    https://github.com/BurntSushi/ripgrep

    - exa: Replacement for 'ls' written in Rust
    https://github.com/ogham/exa

    - bat: A 'cat' replacement.  I recommend following the customizations.
    https://github.com/sharkdp/bat

Written in Python:

    - asciinema: Terminal session recorder
    https://github.com/asciinema/asciinema

    - httpie: Modern command line HTTP client
    https://github.com/jakubroztocil/httpie

    - visidata: A terminal spreadsheet multitool for discovering and arranging data
    https://github.com/saulpw/visidata

    - youtube-dl: Command-line program to download videos from YouTube.com and other video sites
    https://github.com/rg3/youtube-dl

    - pgcli:  Postgres cli with autocomplete and syntax highlighting
    https://github.com/dbcli/pgcli

Written in C

    - jq: Command-line JSON processor
    https://github.com/stedolan/jq

    - tmux:  a terminal multiplexer
    https://github.com/tmux/tmux

    - pspg:  postgres pager (you can combine it with pgcli)
    https://github.com/okbob/pspg

As you are using tmux, also check out the awesome tmuxp written in Python. The author is very thorough with the package; testing and documentation. Tmuxp allows you to quickly launch pre-configured tmux sessions.

Thanks for the tip. I briefly read through the docs and it's not clear what this offers above and beyond what tmux does?

I think the idea is to be able to use tmux in different layouts, configured by simple plaintext. I'll try it out.

The benefit of tmuxp is pretty clear from its README[1]. It gives this example:

    session_name: 4-pane-split
    windows:
    - window_name: dev window
      layout: tiled
      shell_command_before:
        - cd ~/                    # run as a first command in all panes
      panes:
        - shell_command:           # pane no. 1
            - cd /var/log          # run multiple commands in this pane
            - ls -al | grep \.log
        - echo second pane         # pane no. 2
        - echo third pane          # pane no. 3
        - echo forth pane          # pane no. 4

The example is very simple (I use tmuxp to heavily configure the session initialization), but it shows how you can have a config file to set a session's name, set the number and names of windows in there, set the layout of panes differently if you want in each window, run commands like "cd", set env vars differently if you want, in each pane.

You can find a lot more examples of tmuxp config on its doc site[2].

tmux doesn't allow one to have such quick, easy-to-read config files. You can of course do most of this in your .tmux.conf, but it will be too complicated to allow reuse and quick modifications.

[1]: https://github.com/tmux-python/tmuxp#load-a-tmux-session

[2]: https://tmuxp.git-pull.com/en/latest/examples.html

That looks handy, i can see a, 'setup my testing and development environment' with just a quick cp-and-edit of config file.

I use a shell script for such config. eg tmux split-window -h ; tmux send-keys "ls -la" C-m; ...

As I mentioned earlier, it is possible to do such things within the .tmux.conf, but not practical. Here's one of my real tmuxp configs:

    # How to use this file:
    #   tmuxp load /path/to/this/file/main.yaml
    # - Specifying the .yaml extension is optional
    # - Specifying the path is optional if file is in ~/.tmuxp
    #
    #   tmuxp --help for full help.
    session_name: main
    suppress_history: false
    start_directory: '~'
    shell_command_before:
      - unsetenv SHELL_CONFIG_LOADED
      - source ~/.alias
    windows:
      - window_name: '*scratch*'
        layout: tiled
        focus: true
        panes:
          - cd ~/downloads/git           # Top-left
          - cd ~/downloads/git/emacs     # Top-right
          - focus: true                  # Bottom-left
            shell_command:
              - bob
              - sleep 2
              - tclr                     # Clear tmux status indicators
          - shell_command:               # Bottom-right
              - cd ~/.emacs.d/elisp/org-mode
              - module load gcc/6.1.0    # Needed for ob-fortran tests to pass when doing make test for Org
      - window_name: foo-1
        layout: even-horizontal
        shell_command_before:
          - bob
          - cduh
        panes:
          - null
          - null
          - null
      - window_name: foo-2
        layout: tiled
        shell_command_before:
          - bob
          - cduh
        panes:
          - null
          - null
          - null
          - null
      - window_name: uvm_lib
        layout: tiled
        panes:
          - cd ~/gitlab/bar_uvm_lib
      - window_name: sandbox
        layout: even-horizontal
        panes:
          - null
          - null
          - null
      - window_name: hugo
        layout: tiled
        panes:
          - shell_command:
              - cd '~/downloads/git'
          - shell_command:
              - cd '~/e/elisp/ox-hugo/example-site'
              - ./srv
          - shell_command:
              - cd '~/hugo'
          - focus: true
            shell_command:
              - cd '~/e/elisp/ox-hugo'
      - window_name: nim
        layout: even-horizontal
        panes:
          - cd ~/sandbox/nim
          - cd ~/sandbox/nim
          - focus: true
            shell_command:
              - cd ~/sandbox/nim/chat
      - window_name: tok
        layout: even-horizontal
        shell_command_before:
          - tok
          - cduh
        panes:
          - null
          - null
          - null
      - window_name: zoo
        layout: even-horizontal
        shell_command_before:
          - bob
          - cduh
        panes:
          - null
          - null
          - null

I don't do it within the .tmux.conf - I use separate scripts, one per project.

I understand. But the point I am trying to make applies there too. A script to do what I am doing in the tmuxp config I last pasted would be too unmanageable, and not as readable and reusable.

Httpie is really pleasant to use

https://github.com/BurntSushi/xsv is a must if you do anything with CSV.

I wish HN handled long horizontal scrolling lines on mobile better. It’s aggravating that we are a site full of programmers and we can’t get the basics of usability in 2018.

One thing that I think most people don't consider often is that making things more usable isn't always good. Medium and message are interlinked. If everyone in the world switched to mobile, it would change the content of the articles we read and it would change the discussion of those articles. When you make something more usable on a different platform, you're encouraging the biases that the platform compels. In the case of mobile this means encouraging shallower content and briefer discussions, among other things.

https://en.wikipedia.org/wiki/The_medium_is_the_message

You can have different rendering for mobile/non-mobile.

We are in the midst of everyone switching to mobile.

I know. I'm pointing out why this is a bad thing, because most people who advocate for better mobile support on Hacker News don't see themselves as advocating for a reduction in the quality of discussion and a reduction in the depth and complexity of the articles being shared.

Thanks, I see what you mean now. I didn't get it because I use a mobile platform as desktop, but that's not generally true (so far).

Sure making things more usable may sometimes have trade-offs. But I'm struggling to think of the disadvantage of basic markdown support on a site where everyone knows markdown, and even assumes it is supported. People are constantly posting markdown lists that end up all on one line.

I assume the reason it hasnt been fixed is that nobody can be bothered, which is fine. But it's still a little embarrassing. Kind of like how Stackoverflow doesn't support triple backticks. I often see questions with formatting screwed up because people assume a site that heavily relies on markdown supports an extremely common markdown feature.

> we are a site full of programmers

Exactly! And we all know how good most programmers are at UI design ;-)

Thanks for the links. In the future, please don’t use indentation to set off lists like this. It makes it difficult to read due to side scrolling, particularly on mobile. People often use - (hyphen) or * set of bullet points, and separate lines with new lines. You don’t get <li> elements, but it does work. Plus, your links will work, too :)

The problem isn't using indentation. The problem is the platform's rendering on mobile.

Agree on links tho.

Given that we're members of HN (as opposed to someone running the site), I try to provide constructive advice that others can actually use to improve the situation. The admins are well aware of the situation; continuing to argue that it's a problem with the platform while refusing to do what you can to work with the given situation seems like a needless waste of a comment.

Protests sometimes succeed.

Formatting:

Written in Rust

- fd: A simple, fast and user-friendly alternative to 'find' https://github.com/sharkdp/fd

- ripgrep: ripgrep recursively searches directories for a regex pattern https://github.com/BurntSushi/ripgrep

- exa: Replacement for 'ls' written in Rust https://github.com/ogham/exa

- bat: A 'cat' replacement. I recommend following the customizations. https://github.com/sharkdp/bat

Written in Python

- asciinema: Terminal session recorder https://github.com/asciinema/asciinema

- httpie: Modern command line HTTP client https://github.com/jakubroztocil/httpie

- visidata: A terminal spreadsheet multitool for discovering and arranging data https://github.com/saulpw/visidata

- youtube-dl: Command-line program to download videos from YouTube.com and other video sites https://github.com/rg3/youtube-dl

- pgcli: Postgres cli with autocomplete and syntax highlighting https://github.com/dbcli/pgcli

Written in C

- jq: Command-line JSON processor https://github.com/stedolan/jq

- tmux: a terminal multiplexer https://github.com/tmux/tmux

- pspg: postgres pager (you can combine it with pgcli) https://github.com/okbob/pspg reply

Written in Rust:

- rq: A tool for doing record analysis and transformation (replacement for jq, supports yaml, json, avro, protocol buffers, etc.) https://github.com/dflemstr/rq

  history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl |  head -n10
     1	167  16.7%  git
     2	130  13%    ll
     3	94   9.4%   clear
     4	90   9%     make
     5	80   8%     cd
     6	57   5.7%   e
     7	52   5.2%   cat
     8	45   4.5%   man
     9	38   3.8%   valgrind
    10	37   3.7%   rm

Where ´e´ is an alias for ´vim´.

Guessing 'll' is an alias for 'ls -al' or similar as well?

It's a common alias in Ubuntu systems I believe.

I'm just doing a confirmation, since I personally completely overwrite the .bashrc with my own, eliminating any default aliases.

My output from the same:

     1	93  18.6%  l
     2	75  15%    exit
     3	60  12%    zsh
     4	53  10.6%  cd
     5	47  9.4%   nvim
     6	20  4%     ssh
     7	14  2.8%   lt
     8	14  2.8%   ls
     9	14  2.8%   la
    10	10  2%     sudo

Clear? I always press ctrl-l

Me too. But when programming I usually run

  clear && make all && ./a.out

I can copy/paste your statement and it works..

but when I try to save it as an alias method.. I keep getting error:

awk: cmd. line:1: {CMD[]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;} awk: cmd. line:1: ^ syntax error awk: cmd. line:1: error: invalid subscript expression

I thought I would just escape the double quotes ", and things would work. :/ Any ideas?

That's not how shell quoting works. I don't want to explain the mistake and the fix.

Use a shell function instead of an alias, that's the better tool here.

  alias mostused='history | awk '\''{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}'\'' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl |  head -n10'

My version:

     1	178  35.5289%   gi
     2	90   17.9641%   ls
     3	77   15.3693%   vim
     4	51   10.1796%   fg
     5	28   5.58882%   cd
     6	20   3.99202%   jobs
     7	10   1.99601%   date
     8	6    1.1976%    pass
     9	5    0.998004%  docker
    10	4    0.798403%  sudo

I know `gi` looks silly as a `git` alias but it has an ergonomic benefit at least for touch typists.

     1  401  16.04%  git
     2  337  13.48%  fg
     3  241  9.64%   cargo
     4  234  9.36%   cd
     5  197  7.88%   ls
     6  195  7.8%    tig
     7  165  6.6%    task
     8  81   3.24%   vim
     9  38   1.52%   imag
    10  31   1.24%   pwd

"imag" is my own tool, a personal information management suite for the commandline.

Perhaps you want new tabs for your shell instead of typing 'fg' every so often.

Using zsh the equivalent command is ->% zsh_stats Sample: 1 102 15.4545% ls 2 53 8.0303% sudo 3 33 5% vim 4 30 4.54545% python 5 29 4.39394% cd 6 20 3.0303% P 7 18 2.72727% which 8 17 2.57576% pip 9 15 2.27273% touch 10 15 2.27273% la 11 13 1.9697% git 12 12 1.81818% kill 13 12 1.81818% PS 14 11 1.66667% ~ 15 10 1.51515% ln 16 10 1.51515% jobs 17 10 1.51515% cat 18 9 1.36364% take 19 8 1.21212% ~\n 20 8 1.21212% snap

If by terminal programs you mean command-line interface (CLI) programs, then you'll never regret learning how core Unix commands work, in themselves and in pipelines, and their POSIX-defined interfaces. I use these often: make grep find sed xargs m4 cut tr cat tee sort uniq head tail pg/more/less ps pkill/pgrep. My favourite among them is make, once you learn how it's meant to be used, it's an incredibly useful and flexible tool.

Some other useful CLI programs that I use and are not part of POSIX are: strace, perl, webalizer, rcs. RCS might be a mostly-forgotten VCS tool, but it is very suitable for version-controlling projects that consist of single files. I mostly use it for plain-text or Org-mode documents.

If you meant terminal-UI (TUI) programs, like top/htop, vi, etc., I don't really use them. But I've used these in the past:

- mutt: a very nice mail client

- newsbeuter: RSS/ATOM feeds client

- vi: I do occasionally use vi in some situations

In the past years I've started to use Emacs tools instead of these because Emacs is a very rewarding, composable and comfortable working environment. Still, I use CLI programs through Bash/shell-mode all the time, because it's the most efficient way to do many tasks.

The pipe viewer pv: https://ivarch.com/programs/pv.shtml

Great way to give you a progress bar on a long-running process from the command line.

entr is useful for doing stuff on file change:

http://www.entrproject.org/

GNU parallel is like xargs, but in parallel:

https://www.gnu.org/software/parallel/

First of all, you might want to check out the subreddit:

/r/commandline https://www.reddit.com/r/commandline/

general commands

- awk/grep/sed/wc - text file processing

- convert (image magick) - image processing tool

- cpulimit - limit the cpu usage of a process

- fdupes - check for duplicate files and optionally remove them

- pdftk - concatenate pdf files and more - https://www.pdflabs.com/tools/pdftk-the-pdf-toolkit/

- rsync - back up tool

- tmux - terminal multiplexer - almost always my first install

- tree - view directory contents in a tree format

- zcat/zmore - display a file that is compressed (e.g. zmore /var/log/syslog.2.gz)

security/reverse engineering:

- binwalk - search binary images for embedded files

- gdb + peda - debugger with extensions - https://github.com/longld/peda

- nmap - network scanning tool

- radare2 - command line dissassembler - https://github.com/radare/radare2

- scapy - network packet crafting - https://github.com/secdev/scapy

- tshark - network traffic analyzer

- tcpdump - network traffic analyzer

- xxd - hexdump tool

Bonus:

- find - find files - I often use this to find files containing a string:

$ cat bin/findstring

#!/bin/sh

find . -type f -exec grep -i -l "$@" {} \;

If you're just using find to run grep against all files then you would be better off using the recursive flag in grep instead

I like the composition of tools, it lets you extend your usage pretty easily. I personally run this 20 times a day: find -name '*.java' | xargs grep 'pattern'

But at least there you're searching for specific files. I was talking only about running a find to launch an instance of grep for every file found. That's more keystrokes to get a slower result.

For what it's worth, my biggest usage of find is similar to your java example (except a different file extension). I also do do '-exec sed...') a lot too.

if you are piping to xargs, for love of god at least use -print0

With ag:

    ag -G java pattern

grep has to be one of the most useful things i've ever used

fdupes is terrific, but I find that rdfind is an improved version. Can do useful things that fdupes can't.

Pdftk is a must. I use pdflatex also a lot of course. And R and psql.

https://github.com/junegunn/fzf

Fuzzy Finder for files, history and more. Integrates with Vim (and maybe other editors). It is wicked fast.

And if you use the bash/zsh functions the provide in the wiki, it becomes even more useful for git, file opening, etc

I basically live in vim and tmux when I'm doing development work. Tmux IMO basically turns a terminal from a handy utility to a fully productive workspace.

I switched to ripgrep for my grepping needs, I find it fast and the syntax and output coloring handy.

Just found fastmod recently, https://github.com/facebookincubator/fastmod. Very handy for doing bulk substitutions of strings in project directories - it can prompt with a diff for every change and do full regexes, so you can try out tricky things and make sure it isn't doing anything crazy before you let it make 500 changes throughout your repo.

Also a fan of gitsh, https://github.com/thoughtbot/gitsh. Helps out with using Git on the command line just the right amount IMO - just avoiding the git prefix, tab completion, and a little syntax highlighting, instead of trying to redo the whole interface.

I don't use it that much, but jq, https://stedolan.github.io/jq/ is really handy for slicing and dicing json data without diving into the complexity of writing actual programs.

And on OS X, homebrew is indispensable for actually managing all of your CLI programs.

I also have found this Google Drive cli handy, https://github.com/odeke-em/drive. If you want to sync some but not all of your Google Drive files, and only push and pull on command instead of continuously, it's great.

Since iTunes has been getting buggier and buggier for me (and I'm trying to switch to programs that run on all platforms), I've replaced with...

beets http://beets.io/ for library management

mpd https://www.musicpd.org/ for serving music &

ncmpcpp https://rybczak.net/ncmpcpp/ for playing music (which I alias to `music`)

Combine with "abcde" for ripping CDs.

z and autojump are great, but if you want a full fledged file manager you should try ranger:

https://ranger.github.io/

For news, newsboat is quite nice:

https://newsboat.org/

For parsing and select actions over url links you can use urlview (it works perfectly with pdftotext, newsboat, mutt, vim, w3m, etc.):

https://github.com/sigpipe/urlview

You can download webpages with httrack:

http://www.httrack.com/

rsync is a fundamental tool to move files between computers or do backups, forget about scp:

https://rsync.samba.org/

You can work with most pdf and ps (those that are not scanned images) using your usual text tools,

http://www.xpdfreader.com/

There are even ready to go pdf + grep commands:

https://pdfgrep.org/

Somewhat meta, when you live in the terminal you need to be sure that you are a bit POSIX compliant. shellcheck is a good tool for that:

https://www.shellcheck.net/

Finally, there are a two packages that should be in any distributions (include commands like vidir, parallel, watch, etc.):

https://joeyh.name/code/moreutils/ https://gitlab.com/procps-ng/procps/

Some of them that I can't live without are

    - git-standup http://github.com/kamranahmedse/git-standup

    - z https://github.com/rupa/z

    - ZSH https://github.com/robbyrussell/oh-my-zsh

Apart from that, some that I use time to time

    - curl-size https://github.com/egoist/curl-size

    - ripgrep https://github.com/BurntSushi/ripgrep

    - jq https://github.com/stedolan/jq

    - asciinema https://github.com/asciinema/asciinema

git-standup seems cool, but misses the one thing I would want from it: show me the commits I've made from each repo that I work in, sorted by time. That way I can see that I made commit 1 in repo X, then commit 2 in repo Y, then commit 3 in repo X. Also instead of "-d 4" to see 4 days ago, "January 15th" or "Thursday" would be handy.

> show me the commits I've made from each repo that I work in, sorted by time

It does show the commits from each repo sorted by time. Repositories are not sorted though, if that is what you are asking for.

Yeah, I'm only interested in the actual flow of all my work, not the flow of my work in individual repos. I can use 'git log' to list my work in a single repo.

Just notice that you linked to Oh My Zsh (which, granted, is essential) but not to ZSH (http://www.zsh.org/)

>(which, granted, is essential)

Unless you belong to the Prezto or Antigen sects. :)

If you like youtube-dl you should have a look at https://github.com/mafintosh/peerflix !! (:

It is no longer updated, but I recommend https://inconsolation.wordpress.com/ as a very good read. From the about page:

>This blog is intended as a quick look at terminal-based applications available in Linux, and in some similar operating systems. Not only are these ideal for modern, low-power and space-saving computers, but they go a long way toward invigorating out-of-date hardware.

locate may be the most-overlooked timesaver on the GNU distro. For searches of more than ~50000 directories, it's an order of magnitude faster than find:

  $ time locate -0 /{opt,usr}/\*.gitignore |
    xargs -0 stat --format=%s | wc
      387     387    1225

  real	0m0.087s

  $ time find /{opt,usr} -name '.gitignore' -print0 2>/dev/null |
    xargs -0 stat --format=%s | wc
      387     387    1225

  real	0m0.748s

The speedup comes from reading a single database file rather than every directory within the tree to be searched. If the files you want were last touched today, a manually triggered rebuild of the database may be needed:

  $ sudo updatedb

The argument syntax for path specification of locate I find more convenient as well.

Ever since I learned that USENET was still a thing for a current area of interest:

slrn http://slrn.sourceforge.net/ for reading newsgroups

My old fave (which I didn't even remember the name of) was

nn http://www.nndev.org/

... which had too steep a (re)learning curve for reading just the one newsgroup.

Out of curiosity, where'd you sign up?

http://www.eternal-september.org/

Here are some of my favorites that I use daily:

- tmux

- mutt - email client

- wyrd + remind - calendar

- ledger - double-entry accounting tool

- weechat + weeslack plugin - fully featured, super fast slack from the CLI

+1 for remind and wyrd. Remind is really, really powerful. You can express things like an appointment that fires "every 3 weeks starting May 4th, but if that day falls on a weekend or holiday then {skip it, move it to the following/preceding weekday that isn't a holiday}", and lots more. Wyrd is a really nice console frontend to remind (I just found that there are graphical frontends available too). Finally with rem2ics you can express really complicated things in Remind and then export the results as an iCalendar-compatible format and import that into whatever other calendar you want.

I used screen, remind+wyrd, mutt, and irssi (and even ledger) in school; I got a job at a bigco that pretty much required me to use gmail and gcal, and I stopped using console things as much. Lately I've been toying around with moving my mail flow back into console / console-ish inside emacs.

This brings back all kinds of memories: https://www.roaringpenguin.com/wiki/index.php/Remind_use_cas...

Meanwhile on twitter: https://twitter.com/b0rk/status/993165679833567233

https://www.passwordstore.org/ for passwords.

Emacs, tmux, tmuxp, ripgrep, eless[1], ncdu, exa (ls replacement. I often use 'exa --tree'), sed, awk, peco (I use this in aliases to select one or more items from a command outputting multiple values), hugo, dtrx, textract (npm), wget, shellcheck, rsync, htop.

I heavily use git, but via Magit in Emacs.

[1]: My own little utility to use bare bones Emacs as less/pager: https://eless.scripter.co

Wow, peco is so convenient, now I don't have to keep trying to grep every possibilities.

I find myself using noti a lot. It's effectively just a message generator that can receive data over stdout or trigger manually. It uses whatever the most appropriate notification method for your system is, so it feels very seemless. I use it for long-running tasks where I don't really know how long it's going to take.

https://github.com/variadico/noti

CSVkit - http://csvkit.readthedocs.io/en/1.0.3/

A collection of tools for slicing and dicing csv files on the command line.

A few I don't see otherwise listed:

    export MARKPATH=$HOME/.marks
    function jump {
        cd -P $MARKPATH/$1 2> /dev/null || (echo "No such mark: $1" && marks)
    }
    function mark {
        mkdir -p $MARKPATH; ln -s $(pwd) $MARKPATH/$1
    }
    function unmark {
        rm -i $MARKPATH/$1
    }
    function marks {
        ls -l $MARKPATH | sed 's/  / /g' | cut -d' ' -f9- && echo
    }
    _jump()
    {
        local cur=${COMP_WORDS[COMP_CWORD]}
        COMPREPLY=( $(compgen -W "$( ls $MARKPATH )" -- $cur) )
    }
    complete -F _jump jump

I used to use this one more; converted to vimwiki instead: http://todotxt.org/

Local documentation:

    wget --recursive --level=5 --convert-links --page-requisites --wait=1 --random-wait --timestamping --no-parent $1

The only other ones which top the charts are jq, aws, curl, and docker.

  - googler: Google from the terminal
  - Buku: Bookmark manager like a text-based mini-web
  - nnn: The missing terminal file browser for X
  - bcal: Storage expression calculator
  - imgp: Multi-core batch image resizer and rotator
  - keysniffer: Linux kernel mode debugfs keylogger
  - ddgr: DuckDuckGo from the terminal 
  - pdd: Tiny date, time diff calculator

MPS-Youtube is extremely useful if you don't mind the CLI and want to quickly play a YouTube video. translate-shell if you're a language person. Very complete interface to most popular translation services. atools(aunpack, apack, als, etc) for quick archive manipulation if you're lazy.

mpv + youtube-dl plugin are also a nice combination, especially for live streams.

I prefer Streamlink[0] for live streams.

[0]: https://github.com/streamlink

Could you please elaborate why you prefer streamlink over youtube-dl?

I use them both, it is just youtube-dl is more suited for live streams and youtube-dl is more suited for downloading non-live videos.

Since all the good suggestions are taken, I want to mention a couple of diversions:

fortune: prints something silly or thought-provoking. You can get extra fortune cookie files or add your own.

cowsay: prints an ascii-art cow (or other creature) with a speech-bubble filled with the text of your choice. You can pipe fortune into cowsay for extra fun.

    $ fortune | cowsay
     ______________________________________ 
    / Publishing a volume of verse is like \
    | dropping a rose petal down the Grand |
    \ Canyon and waiting for the echo.     /
     -------------------------------------- 
            \   ^__^
             \  (oo)\_______
                (__)\       )\/\
                    ||----w |
                    ||     ||

These days, there's also:

https://github.com/erkin/ponysay

My latest find is to use a combination of weechat and wee-slack for most of my slack usage (it's note quite there when it comes to editing, or in-line reactions). But works well for the chat part:

https://weechat.org/

https://github.com/wee-slack/wee-slack

As a former BitchX and sometime irssi user, weechat feel cozy. And it doesn't eat all your ram just for chat.

The Fuck - https://github.com/nvbn/thefuck

Tetris:

https://github.com/basic-gongfu/cixl/blob/master/examples/ci...

PNG viewer:

https://github.com/basic-gongfu/cximage/blob/master/examples...

Linux/64 binaries may be found here:

https://github.com/basic-gongfu/cxbin/blob/master/linux64

Emacs.

Just ensure you're running a 256-bit color terminal emulator, and for most uses it doesn't even differ that much from the GUI version. Recently I've been doing a lot of work by SSH-ing to my desktop machine from a small netbook and using the terminal Emacs client. If you run Emacs in server mode, the transition is so smooth that at this point, I don't bother getting out of bed if I need to work on something late in the evening, I just take the netbook and SSH over.

If you don't have a reason to use Emacs from the terminal emulator, use the GUI. And for SSH, it's very hard to reject Tramp. One must is to make it less verbose though: (setq tramp-verbose 1). Then it becomes faster (at least in my perception) and way less noisy.

Maybe I didn't word myself clearly enough - my reason for using Emacs from the terminal emulator is working from a small, portable machine on the go, and pick up stuff where I left it. It so happens that my travel machine is a Windows tablet/netbook, and Emacs has some performance quirks there. SSH-ing to a beefy desktop machine (that also runs an instance of the product I'm working on) saves me from fiddling with Tramp, and lets me almost seamlessly switch between travel mode and working on the desktop.

I find using emacs in the terminal less bothersome on osx because of osx's inane out of the box window management; I can replace it completely with managing emacs frames in screen and that experience doesn't change when I move between platforms.

Agreed. I also like to use TRAMP [1] for remote editing files in Emacs: it gets a local copy of a file you're editing so that it only requires a connection when you save/load (as opposed to constantly streaming Emacs over SSH). Not suitable for all tasks, but it means I'm rarely at the mercy of my lackluster internet connection.

[1] https://www.emacswiki.org/emacs/TrampMode

Piggybacking on OP's suggestion of youtube-dl, i have this alias for quickly grabbing the audio-only version of a youtube video. (I use it when I come across a song on youtube that I can't find for sale or streaming anywhere else)

alias dl-audio='youtube-dl --extract-audio --audio-format mp3'

If you have ffmpeg installed, just drop the video ID after dl-audio, and you'll get an mp3 of the highest quality audio stream available for the video.

I wouldn't specify mp3, unless you need to support legacy/device playback - there are often better audio versions available (ogg, aac etc).

- Midnight commander - Best file manager ever

- htop - a graphical top

- xonsh: A shell written in Python. You can now write your shell scripts in Python (with access to any Python library)

- mplayer: Video player

There's so many programs and scripts I use it's hard to keep them all straight. So, instead of trying I just exported a list of every command in my history that I've used at least twice:

https://gist.github.com/jamestomasino/9d912b409e5b6067ac233e...

Also of interest, my custom functions:

https://github.com/jamestomasino/dotfiles/tree/master/bash/....

And finally, my OSX & Linux Mint install scripts which include a lot of great app packages:

https://github.com/jamestomasino/dotfiles/blob/master/osxset... https://github.com/jamestomasino/dotfiles/blob/master/mintse...

ncdu to analyze disk space and find huge folders you can clean

And `z` (the zsh plugin) to cd into your most common directories

Thanks for sharing - I did not know about ncdu.

I typically do a series of "du -s -h * | sort -h" and this simplifies things.

wow, learned about that flag to sort today, thanks. I've been just doing "du -s | sort -n"

fpp http://facebook.github.io/PathPicker/

Rad

micro, mc, ag silver searcher, stat, find, aria2c, xev, w3m, wmctrl, nohup, xdotool, docker, php, ffmpeg, phantomjs, du, ls [From my history file, all others are homemade scripts and little functions in .bashrc]

ncdu is much better than just du.

Thanks. Will try it in details. Currently, it is taking like really forever to scan my bunchs of old school hard drives! du options are easy to remember, ergonomy first^ May I suggest a good one to remember `ls -ltrapR` <- It always kill anyone aside ;)

And it was an amazing jump in productivity when i forced myself to learn `stat` shortcuts. Recommended!

Genuinely the only way I use du is like this:

  du -sh *

which shows total disk usage for every file and folder in the current directory. I've actually aliased it as `dush` because I use it so often.

Bonus tip: if you're using btrfs, it has a built-in du that keeps track of cow statistics, so you can see which files/folders are deduped and which aren't. It's invoked with

  btrfs filesystem du -s *

or (because the btrfs binary understands abbreviations)

  btrfs fi du -s *

tmux https://github.com/tmux/tmux/wiki

glances https://nicolargo.github.io/glances/

rtv https://github.com/michael-lazar/rtv

rainbowstream https://github.com/orakaro/rainbowstream

discurses https://github.com/topisani/Discurses

telegram-cli https://github.com/vysheng/tg

Since a couple people mentioned z and autojump: I recently discovered z and found it so useful I ported it to Windows/Powershell.

https://github.com/dwwmmn/z.ps1

It's a little simplistic for now but I hope it helps someone.

For me personally, I guess netcat, socat, tcpdump and SSH (for tunneling) qualify the most. I'm working in a Windows environment where I'm usually not allowed to install new software, and I miss those by far the most.

Often, I need to do a test run of something which involves slightly rerouting network traffic and I'll spend half a day hacking together some horrible "solution" that I could literally have done in 3 lines of bash code on a UNIX-like system. Maybe throw hexdump for looking at binary stuff in there too.

Edit: Oh, and of course GDB! If only I could have the accumulated days of log forensics back that were done just because it's still faster to do that instead of getting the visual studio debugger to work remotely. Whatever happened to KISS? -_-

For Windows, I'd like to point to http://scoop.sh as mentioned here:

https://news.ycombinator.com/item?id=17019197

It's a bit like a mix of (x)stow and apt for Windows. Or the good parts of arch aur with binary distribution.

Should probably also mention that I'm experimenting with asdf as a version manager - to avoid a long list of setup in bashrc to adjust paths to various interpreters. It follows a lot of best practices that various dedicated version managers seem to pick up eventually:

https://github.com/asdf-vm/asdf

* exiftool - photo metadata editing tool

* dcraw - RAW photo processing tool

* imagemagick - image processing library

* pandoc - convert one file into another

* diceware - random word generator (useful for creating passwords)

* csvtomd - convert csv files to markdown tables

* newsbeuter - RSS reader

* zathura - pdf viewer

* weechat - irc client

* ncdu - interactive disk usage

* ranger - file manager

* mpd - music player daemon

* vimpc - vim inspired client for mpd

bat https://github.com/sharkdp/bat

exa http://the.exa.website

Both are good additions.

Some time ago I wrote just for fun this couple of lines:

> What else I'm missing out on

echo 'Currently there are ' $(ver=(/bin /sbin /usr/bin /usr/sbin) && ls ${ver[*]} | wc -l) 'commands installed and available in the system'

This prints the man page of 3 random commands. One by one. Is a way to discover a few new commands each day. To go to the next command and exit press q. I hope you enjoy it.

commandoftheday=(/bin /sbin /usr/bin /usr/sbin) && ls ${commandoftheday} |sort -R |head -3 |xargs man '{}'

GCE (Global Commands Execution), which is a desktop application that provides a single context to manage and run multiple scripts.

https://github.com/Dewep/GCE

Why?

As web applications are being split in micro services and independant entities, it becomes harder and harder to setup a local dev environment and you have to handle dozen of shell terminals.

GCE aims to take that pain away by offering you the ability to configure the setup of your projects once and run it in the blink of an eye.

A/V: mplayer, aa-xine, surfraw, mpsyt, youtube-dl, cdparanoia, oggenc

Network mgt: ipcalc, mtr, wireshark, nc

Comms & Info: mutt, w3m, irssi, screen (though if you're just starting: tmux), swaks (smtp swiss army knife),

Code & Tasks: make, gawk, sponge (moreutils generally), xclip (pbcopy OSX), at, batch, and cron,

Calculators: units (GNU, not BSD), bc, & dc,

Editing & Docs: pandoc, vim, less ("&" filters!),

Files: curl, wget, scp, rsync, & lftp, mc, column, tree,

Images: mogrify / convert (ImageMagick),

Sysadmin: apt, grep-dpkg, apt-file, auto-apt, pstree,

I just started using cmus and it works great. Also loads up the music library pretty fast.

tmux, neovim are great together (after some configuration.)

Mplayer seems better than mpv. They need configuration though.

htop/top are pretty useful for figuring out which applications are taking up cpu resources.

cpufreq-utils are useful to set max/min cpu speeds if you're running low on laptop battery.

meld is an amazing diff tool though not command line. vimdiff is useful too.

wuzz: https://github.com/asciimoo/wuzz

I love tig https://github.com/jonas/tig

I can't live without `sudo` :p

    - `ffmpeg` one utility for video manipulation to rule them all.

    - `ping` to check whether I am connected to the internet or not.

    - `curl -vvv maktabkhooneh.org` to check where is the connection problem happens.

    - `howdoi` search StackOverflow for snippets to solve your problem from the command line.

- ncdu for disk usage and deleting extra stuff

xrandr: Manege external monitors

zsh: Stepping up your terminal game - https://github.com/robbyrussell/oh-my-zsh

tmux: A terminal multiplexer, aka turns your terminal to a minimalist workspace (the only sane way I found to copy terminal output without using a mouse)

howdoi: Search stachoverflow/stackexchange from the CLI - https://github.com/gleitz/howdoi/

nmtui: WIFI/Networking from the terminal - https://github.com/heftig/NetworkManager

bluetoothctl: Bluetooth from the terminal

___________________

Lists for exploration:

- https://en.wikipedia.org/wiki/List_of_Unix_commands

- https://wiki.archlinux.org/index.php/List_of_applications

- https://github.com/alebcay/awesome-shell

- https://github.com/agarrharr/awesome-cli-apps

- https://github.com/herrbischoff/awesome-command-line-apps

(The github links seems similar, but there are some differences indeed)

Apperanlty I can't make a list of urls on HN, WTF

jq - amazingly useful for processing JSON

Ah, you beat me to it!

REQUEST: jq is great for json - but is there a "jq for xml"?

Some webservices are still only available in xml (e.g. uv levels). I currently use XMLStarlet for xml in the terminal. There are extra annoyances in xml that you can't avoid (like namespaces), but it could be a lot closer to jq.

I’ve used https://github.com/ericchiang/xpup for this a few times.

Thanks, seems the right idea, but browsing the code, decode just interprets a single xpath expression. While that is useful (and I've used xpath from the CL like before), I found I quickly needed to be able to chain them together etc.

pup (https://github.com/ericchiang/pup/blob/master/README.md) which apparently inspired xpup does seem to have this, e.g. chaining, but unfortunately it uses css selectors instead of xpath.

xml2 -- for simple tasks.

pianobar - command line Pandora client

http://6xq.net/pianobar/

https://github.com/PromyLOPh/pianobar

Tmux and ripgrep are two that I use daily. I used to use cloc then switched to tokei, then to my own scc which I use frequently along with lc. Hyperfine is something I have been using a lot and despite having ripgrep I still use grep often with xargs.

`makearms`, a random SVG heraldry generator.

I'm biased, though, because I'm the author. https://github.com/ironarachne/heraldry

rg => (ripgrep) alternative to ack and grep.

There’s also ag (the silver searcher)

I must confess based on the strength of the argument buruntsushi had with the Ripgrep comparison I switched from ag to rg and have not looked back. I was a daily ag user but I don’t need the advanced regex support and I do like the speed.

I love how command-line tools are seeing the same explosion of choices as web-frameworks.

I changed to 'ack' six months ago after having used 'grep' for two decades, and already its successor 'ag' has been made redundant in favor of 'rg'.

My only peeve with 'ack' is that I can't make it switch nicely between paging and not paging:

https://stackoverflow.com/questions/50135983/less-waits-when...

I doubt any of 'ack's competitors have a fix for this, since it seems to be a pager feature that I'm looking for.

Written in Python:

- subliminal: Find & Download subtitles https://github.com/Diaoul/subliminal

I was going to say xterm.. But I guess ssh, git, vim, gcc, make, node, tar, docker, elinks, mutt ls, ps, cp, mv, mkdir, timidity, minicom, tmux, diff, jq and some I can't recall right now

mdp - A command-line based markdown presentation tool - https://github.com/visit1985/mdp

gron, a simple tool to make JSON greppable: https://github.com/tomnomnom/gron

lastpass-cli ("LastPass command line interface tool") [0], with several aliases:

  $ alias | grep lpass
  lp=lpass
  lpls='lpass ls'
  lppw='lpass show -c -G --password'
  lpsh='lpass show -G'
  lpst='lpass status'

[0]: https://github.com/lastpass/lastpass-cli

because I never had to lookup how to use it

EDIT: hm it doesn't even have a man/info page. Where can you learn about "cd" or "cd -", then?!

`cd` is not a command; it is built into your shell.

Some systems, like OS X and CentOS, map the cd man page to builtin which lists all the shell builtins and lets you know you should look at your shell's man page.

man sh

In bash at least, run `help cd`

Try `type cd` and you will see it is a shell builtin. In bash at least you can see the help via `help cd`.

See 1.4.1.7 here for how bash searches for commands: http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_01_04...

cd has at least two man pages on Fedora 27: BASH_BUILTINS(1) and CD(1P) from POSIX Programmer's Manual.

> hm it doesn't even have a man/info page.

It does on Unix and BSD systems.

https://github.com/wting/autojump

Didn't see anybody mention "miller" yet ... pipe/grep/cut/awk etc for CSV files, really rocks.

sponge is great when you want to read and write to the same file:

    cat file.txt | grep -v secret | sponge file.txt

youtube-dl, get_iplayer.

I used to enjoy "orpheus", and MP3 player. That's ncurses, which I think counts. http://thekonst.net/orpheus

musikcube: a cross-platform, terminal-based music player https://github.com/clangen/musikcube

htop and xstarter (disclaimer, I'm the author of the latter)

Damn, xstarter looks neat! I'll try how well it plays with the suckless terminal :)

Thanks! I think it works well with st!

pdftk. If you find yourself needing to extract pages, split or merge pdf documents, this is wonderful.

Split PDF files by pages, blank pages, file size, bar codes, bookmarks. https://www.coolutils.com/download/PDFSplitter.exe

Some less mentioned ones : tig, htop

tig is the best underrated git frontend out there.

- emacs

- mc

- git

- ssh

emacs

screen

notmuch, isync

powertop

parallel, ag

youtube-dl dvdbackup

I like ffmpeg

vifm

byobu

ZSH

ranger mocp rofi feh urxvt

ranger looks awesome... Going to try to integrate that into my workflow. Thanks.

rainbowstream

A python cli tool for twitter

tldr (https://github.com/tldr-pages) since I always forget the damn parameters for tar & it's more convinient than man

like... tmux?

Parse CSV with SQL

nettop

youtube-dl

Add to your repertoire: mpsyt.

pom

Linux has hundreds of them, in the various bin directories. Many are interesting.

This shell snippet lists all the directories in $PATH, using less, so you can page through them:

  for dir in `echo $PATH | sed 's/:/ /g'`
  do
      echo Listing dir: $dir
      ls -l $dir | nl
  done | less

I came across the watch command recently, which seems useful, and wrote something like it in Python:

A Python version of the Linux watch command:

https://jugad2.blogspot.in/2018/05/a-python-version-of-linux...

I've also been writing Python versions of some other Linux commands, and will post about them here some time later.

Also, for saving (and then viewing) man pages (about commands or other topics), without the formatting characters (such as ^H), on some Linux distros, I find this useful:

m, a Unix shell utility to save cleaned-up man pages as text:

https://jugad2.blogspot.in/2017/03/m-unix-shell-utility-to-s...

> Also, for saving (and then viewing) man pages (about commands or other topics), without the formatting characters (such as ^H), on some Linux distros, I find this useful:

^H is one of the two backspace characters (the other being ^?) depending on which VT standard you're configured to use. So you shouldn't really need a dedicate script to do that as it's just a problem between your pseudo-TTY (Linux console) and your terminal emulator (PuTTY / iTerm / xterm / etc).

Thankfully both are easily configurable. However without knowing which terminal emulator you're using I couldn't walk you throw configuring that. In terms of the TTY; you can change the backspace character via the `stty` command:

    stty erase [hit backspace on your keyboard]

Another thought is it might be a case that the TTY is largely configured correctly already but you're overwriting your environmental variables with non-standard values (eg the $TERM var with a value that differs from the actual terminal you're using) which is causing the pager (`less` / `more` etc. `man` will use this for the paging) to break the standard your terminal is expecting. But either way, this is definitely a configuration problem rather than something that should be fixed with additional parsing scripts.

Thanks for your inputs. A few comments:

I first created that script called m years ago on some Unix boxes that I was using. Could have been HP-UX or other version. And I've used it over the years on many Unix and Linux versions that I worked on. It could be that in some cases, the script is needed due to a tty or TERM or other configuration issue. But I'm pretty sure that I've had the need for it (to remove those control characters) on at least some systems where such config was okay. I know this because, while I do tweak env. var., stty and other settings now and then, I do not always (need to) do so, and have still found control chars [1] in the man output, even when redirecting to file. That is why I created the script, because when working on a C, Python or any other project where I need to read man pages, I often like to redirect the man output to a file (in my ~/man dir) and then read them using vi/vim.

[1] See what I say in that post (about m) about nroff and troff.

Also, I do know about stty, have used it for years, although there is less need for it these days. Used to do a lot of tweaking and experimenting with the erase option (for which, BTW, instead of backspace key in your example, we can also write a literal ^h, i.e. caret, then letter h), intr option, onlcr, ocrnl and others. Used to be good fun and sometimes frustrating too, because docs for this areas were somewhat lacking then.

Also see the man page for the col command, which is used in my m script:

http://man7.org/linux/man-pages/man1/col.1.html

I don't make a habit of commenting on shell scripts what I haven't first read. But thank you all the same.

I used to use 'col' alot to resolve the tabs Vs spaces problem with different actors imposing different preferences in their source code. It's a handy tool but often overlooked.

>I don't make a habit of commenting on shell scripts what I haven't first read. But thank you all the same.

Sorry, I should have added "if you haven't already read it", in my earlier comment.

>I used to use 'col' alot to resolve the tabs Vs spaces problem with different actors imposing different preferences in their source code. It's a handy tool but often overlooked.

Interesting, I may not have noticed or known about the tabs option of col. Cool. Seems it is like the entab program in an edition of the K&R C book.

https://www.google.co.in/search?q=kernighan+and+ritchie+c+pr...

Yeah I'm familiar with groff et al. I used to get this issue on old UNIX boxes but not seen it in any of the Linux distros I use daily (not Freebase).

The other thing worth trying is seeing whether your locals are set correctly and/or set the GROFF_TYPESETTER environment variable. Though this might only by GNU specific. Or just redirect the output (as suggested by the other reply) which is how I used to get around the issue on SunOS at least.

I don't think I have any GROFF* variables set.

Thanks again.

> you shouldn't really need a dedicate script to do that

It isn't because the terminal is mis-configured and/or mis-interpreting those characters, it's because these characters have historically been used in a special way by troff's ascii output driver, as used by the man command.

Bold characters are "emulated" by outputting the character itself, followed by ^H, then by repeating the character, and underline is emulated by outputting _ (the underscore character) followed by ^H, followed by the character to be underlined. This is the same way that bold/underline was achieved on a manual typewriter or by old teletype terminals with paper output.

Pagers like "more" & "less" have in-built behaviour that knows how to interpret these sequences and render bold or underline appropriately, and if you "cat" the file your terminal would probably ignore them, but if you open a file with those sequences in a text editor, you're going to see a bunch of unnecessary ^H characters. The OP's script uses the "col" command to remove the unnecessary ^Bs (and the preceding character) that troff has output.

By default, GNU groff doesn't actually output those sequences anymore and uses ANSI escape codes instead. AFAIK many (most?) distributions actually compile out the ANSI behaviour in favour of the old way though (because "more" and "less" don't actually behave correctly with the ANSI characters by default), but some don't, which breaks the script (it's broken in Cygwin, for instance).

FWIW, if you need a fool-proof way to convert a man page to plain old ASCII with no escapes at all, it's easiest just to redirect the output of "man" to a file:

    man ls > ls.txt

The long-winded way (with groff) is something like:

    zcat /usr/share/man/man1/ls.1.gz | groff -man -Tascii -P -cbdu > ls.txt

Ah fair enough. The few times I have seen this issue was when jumping on old Solaris SPARC boxes which also mangled the the backspace key in the interactive terminal. So it sounds like I've put 2 and 2 together and gotten 5.

Though coincidentally I do the redirect trick a lot as was as querying the gzipped raw files too (one of the projects I'm working on requires building a cutdown man page parser).

You described the issue and cause better than I did :)

I didn't know about the ANSI behavior, interesting.

>FWIW, if you need a fool-proof way to convert a man page to plain old ASCII with no escapes at all, it's easiest just to redirect the output of "man" to a file:

> man ls > ls.txt

IIRC, even with that way, the control chars still appeared in the file (at least on some Unix version), which is why I wrote the script in the first place.

> the control chars still appeared in the file (at least on some Unix version)

That's a good point. My whole reply is a bit GNU-centric: the "col -bx" solution should work with everything except groff in ANSI mode (looks like the flag to go back to the standard troff behaviour is GROFF_NO_SGR [1][2], in case anybody is interested).

[1] See: https://linux.die.net/man/1/grotty [2] SGR is "Select Graphics Rendition", apparently: https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_(Select_G...

>Pagers like "more" & "less" have in-built behaviour that knows how to interpret these sequences and render bold or underline appropriately,

Right

>and if you "cat" the file your terminal would probably ignore them

I think the terminal does not ignore them. My guess is that it interprets the characters, but it has no noticeable visual effect on the screen. E.g. if the letter "c" in "cat" is output as "c^hc" (to make in bold in print), the terminal will just print "c", a backspace, then "c" again, which to the user will look the same as a single "c".

The Decorate-Sort-Undecorate idiom (a.k.a Schwartzian Transform) is useful:

https://www.google.co.in/search?q=decorate+sort+undecorate+i...

tig

Link? What's if for?

Its a CLI interface for git.

https://github.com/jonas/tig

Very cool, I like ncurses. Before visiting your link I was like 'isn't git already cli?'

I think TUI as in Text User Interface (similar to GUI) would be a more appropriate term than CLI. git is CLI, tig is TUI and gitk is GUI.

It's a text interface for git. Great for quickly navigating around a repo and, the thing I use it for most, staging individual lines/hunks

https://github.com/jonas/tig

That is all.

Reply on news.ycombinator.com