CLI: Improved
remysharp.com
remysharp.com
htop is also pretty much a great replacement for top. And ripgrep a great replacement for the find | xargs grep pattern.
Aside from that, I'm pretty content with the Unix userland. It's remarkable how well tools have aged, thanks to being composable: doing one thing and communicating via plain text.
I'm less happy with the modern CLI-ncurses userland. Tools like e.g. mutt are little silos and don't compose that well. I've migrated to emacs, where the userland is much more composable.
If programs used typed data, they'd still need the option to output text to present results in a format the user can understand. To do this, a negotiation protocol could be established, like skissane said. This, in my opinion, is BAD, because then there's the possibility or probability that they'll be differences in the information conveyed in the different formats.
I believe that the use of plain text as the universal format of communication between programs is one of the greatest design decisions for Unix and CLI.
You'd standardize it once via an RFC and you'd be done with it.
I don't think your problem is as big as you say it is.
The real problem is that this pile of code we already have kind of works and it's already making trillions for its users. Changing the whole ecosystems would cost millions and millions, for only a very long term and unclear benefit.
In other words, worse is better.
As oblio said, you can come up with a standard conversion of structured data to text. The other way round you need write a parser for every textual output format, and typically people come up with fragile ad-hoc parsers that don't deal with edge cases properly.
Of course, who am I kidding, in real life we have some sort of crappy text interface which is half-baked both for humans and for machines. But we've been using it for almost half a century and it's too widespread to redo, so there we are, plowing through it daily.
1) We don't need to depend on each individual program's programmer to present every control and information consistently between the 2 interfaces.
2) Automation matches normal, manual use. Just put what you normally do on the command line in a file and you're done. There's no need to look through documentation on how to do what you so frequently do, only in a manner that you rarely do.
Also there is no need really to change anything (as in, breaking existing scripts). Selecting a different output format can simply be a command line option. Many tools already offer Json, XML or CSV output. But since development of those tools is so decentralized you'd be hard pressed getting them all to agree on one. But theoretically you can pick any tool you want right now, add --json support and submit a patch.
Are TUIs like htop, tmux, vim, emacs, less, etc. going to be impossible now, or will you do the negotiation protocol? Both options suck.
When programs have both normal output and errors intermixed are the objects going to be intermixed in the output that's presented to the user? For example, if you do a `find /etc/pacman.d`, instead of:
/etc/pacman.d
/etc/pacman.d/mirrorlist.pacnew
/etc/pacman.d/gnupg
/etc/pacman.d/gnupg/trustdb.gpg
/etc/pacman.d/gnupg/crls.d
find: ‘/etc/pacman.d/gnupg/crls.d’: Permission denied
/etc/pacman.d/gnupg/.gpg-v21-migrated
/etc/pacman.d/gnupg/private-keys-v1.d
find: ‘/etc/pacman.d/gnupg/private-keys-v1.d’: Permission denied
/etc/pacman.d/gnupg/tofu.db
/etc/pacman.d/gnupg/openpgp-revocs.d
find: ‘/etc/pacman.d/gnupg/openpgp-revocs.d’: Permission denied
/etc/pacman.d/gnupg/gpg.conf
/etc/pacman.d/gnupg/secring.gpg
/etc/pacman.d/gnupg/pubring.gpg~
/etc/pacman.d/gnupg/pubring.gpg
/etc/pacman.d/mirrorlist
[
"/etc/pacman.d",
"/etc/pacman.d/mirrorlist.pacnew",
"/etc/pacman.d/gnupg",
"/etc/pacman.d/gnupg/trustdb.gpg",
"/etc/pacman.d/gnupg/crls.d",
[
{
type: "Permission denied",
path: "/etc/pacman.d/gnupg/crls.d"
},
"/etc/pacman.d/gnupg/.gpg-v21-migrated",
"/etc/pacman.d/gnupg/private-keys-v1.d",
{
type: "Permission denied",
path: "/etc/pacman.d/gnupg/private-keys-v1.d"
},
"/etc/pacman.d/gnupg/tofu.db",
"/etc/pacman.d/gnupg/openpgp-revocs.d",
{
type: "Permission denied",
path: "/etc/pacman.d/gnupg/openpgp-revocs.d"
},
"/etc/pacman.d/gnupg/gpg.conf",
"/etc/pacman.d/gnupg/secring.gpg",
"/etc/pacman.d/gnupg/pubring.gpg~",
"/etc/pacman.d/gnupg/pubring.gpg",
"/etc/pacman.d/mirrorlist",
]
]
You could have every function incorporate their errors into their normal output, but that means giving up a standardized way of working with errors and warnings. I don't know if you know this, but when you do substitution or piping, by default, only stdout is used. That means that we you do piping, the programs in the pipelines normally do not see the errors in their inputs, and the errors of the multiple concurrently running programs are shown to you intermixed while the pipeline is working. That's a friggin' incredible effect that came from simple design, but when each one would output objects, you'll could get syntax errors or a completely different object like what happened in the above example. You could say, "well, only make stdout an object and let stderr be text," but the fact that they're both the same type means that you can work with the error or only with your errors in pipelines and other shell constructions. For example, `find /etc 2>&1 >/dev/null` will output the directories in /etc you can't read for whatever reason. You might want to pipe that to `xargs chmod` (for whatever reason) after preparing the output to only include the paths.
Right now, programs can strike a good balance in presenting its output in a format that is both readable to humans and other programs. By forcing their output to be structured as objects, and not giving them the option of presenting 2 formats (because we don't want that either), you're removing their ability to present the output in a manner that is readable to humans.
Take for example, rspec's output (a unit testing framework):
$ rspec spec/calculator_spec.rb
F
Failures:
1) Calculator#add returns the sum of its arguments
Failure/Error: expect(Calculator.new.add(1, 2)).to eq(3)
expected: 3
got: nil
(compared using ==)
# ./spec/calcalator_spec.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.00131 seconds (files took 0.10968 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/calcalator_spec.rb:5 # Calculator#add returns the sum of its arguments
The main thing is, though, that, in the question of what the universal format of communication between programs written in different languages should be, text is the simpler, more natural choice over objects. Take note, I don't mean easier. The fact that it's easier is merely coincidence. Simplicity leads to good design because it means less arbitrary choices to make. Less controversial choices to make. Choosing objects leads to more questions: What should the primary types be? Should arrays/lists allow multiple types of elements? Floating types or decimals? Precision restriction on the decimals? Should integers and numbers that allow fractional parts be the same type or different? Should we have a null type? Should we have a date primary type? What about a time primary type? What about a datetime primary type? Whatever answers you give, there will always be groups of people that will dislike them. When you chose text, the only question is really, what encoding? Utf-8. done. Natural, simple design is what we want to be the foundation that myriads of programs and languages can base themselves on and depend on.
There's only one way I'd agree with you that structured output would be nice, and that's with mono-language OSes, like a lisp OS or some other OS where all code is in the same language, and there would be no concept of programs or shared / dynamically loaded libraries or such. In an OS like that, every function is a program, and your shell is the language's REPL. This is bliss when the OS is done in your favorite language. The problem with these kinds of OSes is that we don't all like the same languages and so it'd lead to ridiculous situations where we'd translate a new language into the high level language of the OS. That's what we do in the OS known as the web browser and why we're coming up with WebAssembly.
In conclusion, multi-language OSes like those that are Unix based are awesome, and text as the basis of communication in multi-language OSes is awesome. Therefore, text as the basis of communication in Unix is awesome. :)
If you think something like this is easy much less "you standardize it once and you're done", then you are only cheating yourself out of an essential life lesson.
I’d like to see something like cvs used more, where it can handle the edge cases without breaking, but still doesn’t need a translation step
I have suggested this before: https://news.ycombinator.com/item?id=14675847
(But I don't really care enough about the idea to try to implement it... it would need kernel changes plus enhancements to the user space tools to use it... but, hypothetically, if PTYs got this support as well as pipes, your CLI tool could mark its output as 'text/html', and then your terminal could embed a web browser right in the middle of your terminal window to display it.)
ssh carthoris ls /mnt/media/Movies | grep Spider
(this is just an example). Note that in this example, we have two processes running on two different machines. Indeed, the OSs and systems on these machines may be, um... different. Indeed, I routinely include "cloud" machines in pipelines. Indeed, with ssh, the -Y (or -X) option can introduce a GUI to a part of the command.
I have wished that shar was part of SUS. Also, I find that "exodus" is useful (across Linux anyway -- the systems have to be "reasonably" homogenous). https://github.com/intoli/exodus
In order for this to work over SSH, the SSH client and server would need to be enhanced to exchange this data, and also an SSH protocol extension would need to be defined to convey it across the network.
One might define IOCTLs that work on pipes and PTYs to send/receive control messages. So sshd would read control messages from the PTY and pass them over the network, and the SSH client would receive them and then pass them on to its own stdout using the same ICOTLs. (Alternatively, one might expand the existing control message support that recvmsg/sendmsg supply on sockets to work on pipes and ptys as well.)
Any program supporting such an out-of-band signalling mechanism would have to gracefully degrade when it is absent. If your SSH client or server, or some program in your pipeline, or your terminal emulator, etc, doesn't support them, just fall back on the same mechanisms used today to determine output/input formats.
(rsh is such a deprecated protocol, there would be no point in trying to extend it to support something like this.)
Ha ha, I had dreamed up something like this in one of my wilder imaginings, a while ago: A command-line shell at which you can type pipelines, involving some regular CLI commands, but also GUI commands as components, and when the pipeline is run, those GUIs will pop up in the middle of the pipeline, allow you to interact with them, and then any data output from them will go to the next component in the pipeline :) Don't actually know if the idea makes sense or would be useful.
And another apparent one is image viewer or editor.
There seems to be some precedent for this sort of thing. For example, DVTM can invoke a text editor as a "filter", where the editor UI is drawn on stderr and result saved to stdout.
I did some experimenting recently and found you can open a new /dev/tty file descriptor and tell curses to use this, the rest of the application can continue reading stdin and writing stdout as normal.
The UNIX style.
echo -n 'string' | md5
$string = "string"
$md5 = new-object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$utf8 = new-object -TypeName System.Text.UTF8Encoding
$hash = [System.BitConverter]::ToString($md5.ComputeHash($utf8.GetBytes($string)))
$hash = $hash.ToLower() -replace '-', ''
You can do the same on PowerShell and write a pipeline just the same way.
The best alternative I could find is some community maintained PowerShell extension (with just 177 GitHub stars now), which is far better but the lack of interest in making PowerShell act more straightforward is weird.
"string" | Get-Hash -Algorithm MD5
(https://github.com/Pscx/Pscx)
Still feels funny to read the GGP's comment.
> 70s-era tooling is inferior in some way to something designed with 30 years of hindsight
- structured data
- ability to call any public function across dynamic libraries, COM objects and .NET frameworks
- Powershell modules can also be called as plain .NET code
Which UNIX shell provides this?
$exclude = @("main.js")
$excludeMatch = @("app")
Get-ChildItem -Path $from -Recurse -Exclude $exclude |
where { $excludeMatch -eq $null -or $_.FullName.Replace($from, "") -notmatch $excludeMatch } |
Copy-Item -Destination {
if ($_.PSIsContainer) {
Join-Path $to $_.Parent.FullName.Substring($from.length)
} else {
Join-Path $to $_.FullName.Substring($from.length)
}
} -Force -Exclude $exclude
find sourceFolder \! -path */app/* | xargs cp -t destFolder
rsync -a --exclude=app/ --exclude=main.js sourceFolder/ destFolder/
This example has targeted a very specific deficiency in the way Copy-Item works. I could similarly point out that the following Powershell command would be much more difficult in standard unix tooling:
Get-WMIObject -Computer $remote -Class Win32_Product | Where-Object { $_.name -like "7-Zip*" } | select Version
What do you use for reading email in Emacs?
I like it a lot due to its clever architecture. It never ever touches your email. It operates on a separate tag database.
Then, it's the task of a backend to translate tag changes into maildir actions before and after syncing email. Keeping tag to actions decoupled from the GUI is extremely clever, because it allows implementing basically any email workflow you can imagine.
For simple workflows, calling a one liner notmuch command is sufficient. You don't really need to implement anything.
Mu4e is an alternative client to Notmuch. Quite similar to Mutt. Gnus is the other big alternative. It's quite old, and complex to configure. Besides, the codebase is overcomplicated as it tries to do email in a news-like fashion. Still, it has lots of great ideas on how to deal with email from many sources. E.g. using predictive scoring.
With notmuch and mbsync I have the best email setup I ever have. I wish I'd done it years ago.
My crontab (https://github.com/NateEag/dotfiles/blob/master/lib/.crontab) runs a check-mail script (https://github.com/NateEag/dotfiles/blob/master/bin/check-ma...) every five minutes.
The check-mail script is just a wrapper around mbsync (http://isync.sourceforge.net) that invokes 'notmuch new' after running, so notmuch can index and process new messages.
My notmuch post-new hook does a bunch of tagging for me, so I have to actually look at as little email as feasible, but the main thing it sounds like you're interested in is the notification setup:
https://github.com/NateEag/dotfiles/blob/858cabd9436d377f8fc...
With that, I can batch-process email a few times a day, while staying responsive to any discussions I actually want to be interrupted for.
The missing piece is reasonable logic for knowing when I should be notified of new threads. I'm currently in a job where people don't expect insta-responses to email, thank God, but I've been in ones where they do and I'd have to think about how to handle that more.
For reading and writing email, I just use notmuch's Emacs modes. You can see most of my interesting config here: https://github.com/NateEag/.emacs.d/blob/75d117befeaaab37cc7...
My one annoyance when reading email is that large inline images aren't auto-resized to fit. They should be, but the Emacs build I use doesn't have ImageMagick support compiled in.
My custom.el probably has some notmuch settings too.
That's the highlights. Hope it was helpful.
I'm also looking for a scriptable, text-based RSS reader: https://community.codeselfstudy.com/t/text-based-rss-atom-fe...
Fresh mu/mu4e user here. The advantages over mutt/neomutt are that your e-mail now becomes the part of the same consistent UX of Emacs, and then every improvement to any of your workflows in Emacs is automatically inherited by your e-mail workflow. This depends on how much you like customizing Emacs and/or writing Emacs Lisp to solve your problems. Some practical examples include:
- org-mu4e & org-notmuch will let you link directly to your e-mail messages from your org-mode files; potentially useful if you're using org-capture to quickly add TODOs and notes.
- it's trivial to add any kind of template responses, template subresponses, etc.
- you can make Emacs automatically do things in response to particular e-mails, or you can compose/send e-mails directly from any elisp code
Emacs is a fully programmable environment with lots of existing software packages and the best interoperability story I've ever seen.
I usually just run --citation once on a new system, and never see the notice again, and I have never had the notice cause any problems as it only shows if the output is to the screen.
I prefer PowerShell in this respect where the output of each command is not text streams (as in the Unix world) but objects which can be operated on in a more object oriented way. You spend less time thinking about text parsing and more time thinking about the data you're working with.
I think the command line is great as a way of manipulating data streams but it is incredibly lacking as a user interface. There is very little consistency of the interface between commands and new commands and options aren't easily discoverable.
Huh, TIL. It's just never occurred to me to even bother checking the help for cat.
$ cat --help
Usage: cat [OPTION]... [FILE]...
Concatenate FILE(s) to standard output.
With no FILE, or when FILE is -, read standard input.
-A, --show-all equivalent to -vET
-b, --number-nonblank number nonempty output lines, overrides -n
-e equivalent to -vE
-E, --show-ends display $ at end of each line
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output lines
-t equivalent to -vT
-T, --show-tabs display TAB characters as ^I
-u (ignored)
-v, --show-nonprinting use ^ and M- notation, except for LFD and TAB
--help display this help and exit
--version output version information and exit
Examples:
cat f - g Output f's contents, then standard input, then g's contents.
cat Copy standard input to standard output.
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Full documentation at: <http://www.gnu.org/software/coreutils/cat>
or available locally via: info '(coreutils) cat invocation'I know why the authors chose to use a non-zero return code as the default, but when I'm using grep deep in a pipeline and doing my own checking of the results to see if nothing was found, I don't need grep bombing out the whole pipeline with a non-zero return code.
The alternative of being forced to use "(grep pattern||true)" instead of a plain "grep -q" is kinda painful.
$ cat > foo
here's
some
random
text
$ grep -v some foo; echo $?
here's
random
text
0
$ grep -v bar foo; echo $?
here's
some
random
text
0
$ grep -vE "(here's|some|random|text)" foo; echo $?
1However, from my own observations (and my own use), folks seem to appreciate having some amount of coupling and integration here. I've often seen folks claim that a legitimate use case (for them) for ripgrep is to "replace crazy 'find ./ ... | xargs' concoctions." The degree to which the aforementioned is "crazy" or not varies based on the individual, but there's a not insubstantial number of people who appreciate the succinctness that coupling brings.
As the maintainer, the coupling is annoying, because it means ripgrep needs to implement more stuff. For example, ripgrep provides a `--sort-files` option, which is something that standard grep doesn't need because you can fairly easy compose its output with 'sort'. You could do the same with ripgrep (ripgrep supports the standard composable grep output format), but then you lose the "pretty styling" that users appreciate. So you have no choice.
In terms of giving more structure to output, I am mostly unconvinced by your argument, but I've been happily using text streams as my user interface for a very long time, and I like its balance.
With that said, the next release of ripgrep will come with a --json flag, which enables structured output. :-)
Last year I wrote a friendly script to search content on YouTube, which prints human-readable content if plugged to a terminal and URLs otherwise, and another one which applies a pattern to turn such YouTube URLs to the underlying video or audio streams using youtube-dl. (I don't think this plays along nicely with YouTube's ToS but whatever I'm the only user.)
The obvious use case is to look at the output of the first command, and then pipe it to the second after choosing what to watch, and find a way to feed that to mplayer. So you'd want to provide a shortcut script, maybe make it interactive… but when does it go from an alias to a new program altogether? This interaction is very intuitive in the browser and I find trying to reproduce it with a CLI tool is quite challenging.
Btw I love ripgrep, it made me look very hip in front of colleagues a couple of times.
;)
If you aren't proficient or have an aesthetic or religious aversion to unix userland and traditional tools you'll play some other game I guess. Reinventing the wheel without understanding the model and power is a next-gen game. I don't have time for it.
To be clear, with the current release of ripgrep, you cannot create structured objects from its output as easily as you might think. I get that it's fun to show how to do it with long shell pipelines for simple cases, but the current release of ripgrep would actually require you to parse color escape sequences in order to find all of the match boundaries in each line. This is what tools like VS Code do, for example. The --json output format rectifies that. There are other solutions that might be closer to the text format, but they're just more contortions on the line oriented output format that aren't clearly useful for human consumption, and it's much simpler to just give people what they want: JSON.
Unix shell, on the other hand, is a trade-off: it offers you options to process and automate and reasonable convenience when working interactively. None of those aspects is perfect, of course, but it allows gradual learning curve (i.e. being able to do quick and dirty things from the beginning), is much more versatile and ubiquitous.
On Windows, I strongly prefer PowerShell for interactive work, although I cringe a little each time I see how much memory it uses.
In a programming language, it's obviously no context. But in a shell, I want to spend more time doing and less time reading.
Or not. Take `ps`. You'll need to spend time in documentation anyway, figuring out what process properties can be shown with what flag, and then you'll be bitten later by things like the difference between `ps ux` and `ps aux` including process names in square brackets, etc. Contrast with PS equivalent, `Get-Process`. Type `Get-Process | Get-Member` to list properties of the objects returned by `Get-Process`, and you can quickly see both properties you can inspect (with descriptive names, not "VSZ" or "RSS") and what methods you can call directly (instead of extracting properties and piping to other programs).
This is IMO much cleaner, easier to work with interactively (properties instead of constant parsing and unparsing of text), better for interoperability (you're limited by what actual objects expose, not by what pieces of them a CLI program wishes to print, and if it so happens that objects somehow print more than they expose in properties, you can still call ToString() on them and get that data "the UNIX way"), and correctly separates presentation from content.
The only real drawback I've seen of Powershell is the lack of quality-of-life scripts and executables in the system. Like the md5 example elsewhere in this thread.
Powershell is too complex to be a good shell, and there are too many bizarre idiosyncrasies about it to hold its own as a programming language. It just doesn't really have a place.
I suppose it's just a generic colorizer tool that takes a configuration file of expressions and arbitrarily colors data feed through it?
Examples:
- bash/zsh/fish bring suggestions/autocompletion
- fzf brings fuzzily-matching selections to the terminal,
- https://github.com/ericfreese/rat brings the idea of "widgets" or "layouts",
- https://github.com/mrnugget/fzz brings "preview as you type"
I'm looking for more patterns like that, mainly to explore how to make them work in the terminal and see whether that actually has an impact on everyday actions people perform in the terminal many times per day.
The goal is not to save time, but to reduce mental friction.
I think VSCode uses ripgrep too.
If they choose to approach some problems with different semantics, I would not recommend to alias the new tool over the old one. Just treat it as a like a separate tool and in most cases their new names are as comfortable to type as the 'legacy' tool's. The one exception that comes to my mind is 'exa' vs. 'ls'. Typing 'ls' is a single action for both hands while 'exa' has to be typed with the left hand alone (on QWERTY/Z layouts).
Edited to change: like them to: like ls/find and grep
which is what I should have written originally.
https://github.com/thibran/exa/tree/time_style_hide
Example output of 'exa -l --time-style=hide':
drwxr-xr-x - foo Desktop
drwxr-xr-x - foo Documents
drwxr-xr-x - foo Downloads
drwxr-xr-x - foo Music
drwxr-xr-x - foo Pictures
drwxr-xr-x - foo src
drwxr-xr-x - foo Videos $ l (exa --tree --level 1)
$ l2 (exa --tree --level 2)
Another one potentially for this list is tokei[1] I was trying to count the code in our repos at work and used the venerable 'cloc' utility. It took over 5 minutes. Looking around I found tokei, written in rust. Same-ish results (more accurate actually) took 10 seconds.
We have a whole working group this year focused on making the experience of writing CLIs awesome, so hopefully we’ll see even more great tools in the future!
I need to try implementing the string thing. It'd be a lot more fun to try to compete on speed if we were the same on accuracy.
Can you explain what you mean by this? Something like the people are going to focus on language and library features that help with writing CLIs? Or something else?
Asking because CLIs are one of my interests.
You can reach out to us on gitter [3].
[0]: https://github.com/rust-lang-nursery/cli-wg/issues
[1]: https://github.com/rust-clique/ and https://github.com/assert-rs/
[2]: https://rust-lang-nursery.github.io/cli-wg/index.html
[3]: https://gitter.im/rust-lang/WG-CLI
The areas we're trying to finish up for Rust 2018 Edition:
- Get clap (arg parsing) to 3.0 [4]
- Get assert_cmd [5] / assert_fs [6] to 1.0
- Finish work on man-page generation [7]
- Make it easier to package binaries [8] and document the CI for it [9]
[4]: https://github.com/rust-lang-nursery/cli-wg/issues/41
[5]: https://github.com/assert-rs/assert_cmd/
[6]: https://github.com/assert-rs/assert_fs
[7]: https://github.com/rust-lang-nursery/cli-wg/issues/38
Rust has different community working groups that focus on improving the Rust ecosystem in different ways this year: https://internals.rust-lang.org/t/announcing-the-2018-domain...
The TL;DR is that loc is faster by a few hundred milliseconds depending on repository size, but as cgag mentions doesn't have comment in string detection so can be quite off in its metrics, for example on the Rust repo Tokei says it has 643,754 lines of code where as loc says it's 635,849.
[0]: https://github.com/Aaronepower/tokei/blob/master/COMPARISON....
New and not-popularly adopted languages tend to have more senior people learning and developing software using them; this leads to people (read: recruiters) looking for the people with experience.. People equate the quality of software with something innate with the language, or that the developers are exceptional. Then after being popularly adopted it slumps in perceived elegance and goes on a downward spiral until it ends up like PHP, Ruby or JAVA. (not to undersell the folks who predominantly use those languages, I'm just picking languages that I've seen follow this pattern)
Also, I don't know Rust (but have read that it is somewhat difficult to learn); for the basics, FreePascal (FP) may be easier than Rust to learn and start using, because although it (FP) has advanced language features, for many basic CLI programs, the simpler procedural features should be enough.
Do you have any docs on what you mean by extensibility here? I’m curious!
[0] https://hexdocs.pm/mix/Mix.Task.html [1] https://github.com/jhartwell/Plsm
GCC's static linking issues with glibc don't apply to other compilers.
You just have to look properly for those options.
The only thing is that not all of them are free beer.
The name may not have aged well...
I manually counted a few files for comparison, I believe tokei doesn't understand python-doc-comments and think it's code.
Would you suggest me to learn Rust in order to write CLI programs? I'm also looking at haskell for the same but after this thread, I'm really thinking of going the rust way. Ideas?
I don’t think of Haskell as the best language for command line dev, but I’m far from expert in either language.
I interact with hundreds of servers on a day-to-day basis, and I don't want to go around installing random tools on my servers. But I guess it's an idea for some tools to add to my Ansible server provisioning script :-)
Besides, why not just make a $HOME/bin/ and throw your new stuff in there?
IIS has significant market share. I'll bet most web developers who deploy on it get away without having to use the command line interface very often if at all.
https://news.netcraft.com/archives/2018/08/24/august-2018-we...
They recently finally switched to git for source control, and they want to do everything via GUI. I've tried to show them how some things are just easier/better from the command line, but if something isn't doable via GUI, they won't do it. One of them keeps committing line endings differently than everyone else, and he literally won't run git config --global core.autocrlf true because... I dunno why. He just doesn't want to, because it's the command line.
The only exceptions I can think of are interactive rebase which has a weirdly good command line interface and is really confusing in every GUI I've tried; and continuing/aborting rebase and cherry picks when there is a conflict. And that's only because most GUIs don't bother to actually implement that properly and mislead you into thinking that you should make a new commit.
There are a few things I don't like, like how `git blame` requires me to put my terminal in fullscreen to see the output properly. I wish there was an option to make the output group changes of a commit together. It could put the commit details on a line before and add a single character prefix to all lines to differentiate between file lines and commit lines, just like how ag/ack/rg improve grep's output format for human viewing.
There's also a long-standing bug in `git log --graph` that causes the lines of the graph to sometimes move back to the previous line at the end of a commit description.
I love the -p/--patch option that's available in many subcommands. It's really quick to work with.
What specific things do you not like about git's CLI?
Besides the inconsistencies, there are many gotchas which keep tripping developers. For instance, git pull always tries to merge in the remote branch, but you almost never want that. You can do: git pull --ff-only, but most developers I know don't know that and end up with a mess they have to spend time cleaning up.
The main issue is that git commands mix up so many concepts and the defaults are almost always useless. For instance, git add manages tracking files and staging commits. git rebase both deals with rebasing and history clean-up (rebase -i).
A better CLI would just have consistent clear verbs that expose the git model properly instead of mixing up concepts:
git track <file>
git untrack <file>
git stage <file>
git unstage <file>
git commit
git config --global pull.ff onlyThe biggest for me is staging lines. In GitHub desktop you click the line numbers, in the command line (iirc) you navigate through chunks as they appear in the file, maybe splitting them into smaller chunks and hopefully can get it down to what you want.
# To remove '-' lines, make them ' ' lines (context).
# To remove '+' lines, delete them.
# Lines starting with # will be removed.
>One of them keeps committing line endings differently than everyone else, and he literally won't run git config --global core.autocrlf true because... I dunno why. He just doesn't want to, because it's the command line.
It sounds like someone needs to have a conversation with this engineer. I personally wouldn't want a single engineer on my team or company that has a resistance to learning new skills. Learning new things and better ways to do those things is basically the job description. Hopefully they have a good reason other than being obstinate otherwise they might need to find a new company that tolerates mediocrity :/
It is so much better just to use standard IDE workflows.
https://github.com/wting/autojump
I also use `loop`, my own Rust-based replacement to bash's native loops:
I'd be remiss not to mention pazi however, which is the tool I wrote to replace fasd in my own workflow: https://github.com/euank/pazi
It's similar to autojump, but much faster: https://github.com/euank/pazi/blob/master/docs/Benchmarks.md...
Oh, and it's written in rust :)
$ cd ~/go/src/github.com/my/project
$ kd awesome_project $PWD # create bookmark
$ kd aw # BAM
$ # yay, straight to my project!
$ cd app/controllers/whatever/deeply/nested
$ kd # back to the project root!
$ make
You can even press 'd' twice, and you'll get something similar to prettyping, but you get it for each hop along the path.
Cat is just a tool to dump files to stdout and maybe concatenate them. If you want paging, syntax highlighting, etc, you probably want a tool to replace `less`, not `cat`.
[1]: https://github.com/dguo/dotfiles#replacements-for-unix-comma...
Is Rust a particularly good fit for CLI tools like these ? Why so ?
https://apple.stackexchange.com/questions/47801/is-there-any...
For example, for any diff'ing or cat-like need, I find it SUPER handy to just pipe stdout to vim like so:
$> thing-with-output-that-needs-navigating | vim -
I'm not an Emacs user, but my impression is that over in that world, the shell integrations are crazy good.
Let me be clear: Ponysay has its niche. But with the Ansible stack providing great out of the box integration with Cowsay, it's clear which is the tool of choice for modern workflows.
On a serious note, TL:DR is amazing in some cases. The man page for "tar" compared to the TL:DR is much more unwieldy for 99% of use cases, for example.
I love ripgrep and htop, and I think that a lot of people new to *nix cli's would love tree as well!
>We are reminded that ls isn't the place for code to break a single column into multiple ones, and that mailnews shouldn't have its own more processing or joke encryption code.
The downside being it requires the kitty terminal as it uses features not present in other terminals.
I do have a huge .vimrc and some bash niceties, but I found it better for me to exercise some customization discipline overall.
- percol and ripgrep as alternatives to grep
- tig as TUI for git
- kakoune as alternative to vim
$ filewatcher '**/*.js' 'jshint $FILENAME'
You can do this with `less -F <file>`. It will show the file on the screen like `cat` unless it is longer than one screen, then it will go into paging mode and allow search with `/`.
Side note, how do you make his nice-looking prompt?
cat mysoftware.tar.gz mysoftware.sig > mysoftware.bin
Or alternatively:
cat a.log b.log c.log > monolith.logI'm using pandoc to convert it to groff and read it like a formatted man page, which is ok but not great as it loses some formatting.
pandoc -s -f markdown -t man markdown.md | groff -T utf8 -man | less -R #!/bin/bash
# ~/.lessfilter
case $1 in
*.md) view-md $1;;
*.json) jq -C . $1;;
*) pygmentize $1;;
esac
#!/usr/bin/env ruby
require 'tty-markdown'
puts TTY::Markdown.parse(ARGF.read, colors: 256)
I don't think this was a bug; I think the issue was that htop uses needed something in task_for_pid that required root.
npm outdated --json | ramda --raw-output 'map (.latest)' to-pairs 'map join "@"' alias o="fzf --preview 'bat --color=always {}' | xargs -i '{}' xdg-open '{}'"EDIT: I just saw your comment where you stated you're the author of rg. I honestly haven't tried it before until now. It seems you also chose to output line numbers by default in that case, but it's nice that you have -N. If you don't mind me asking, do you have an option like ag's undocumented -W which allows specifying a max line width for which a matching line is displayed? Regularly when searching a hierarchy a match happens on a generated file where everything is on a single line and so my terminal prints the millions of characters line. -W displays an bracketed ellipsis once the max width is reached on a line.
Right. When you run ripgrep with its output connected to a tty, then its output is "prettified." That means results are grouped by file, colorized and include line numbers. But if you aren't connected to a tty, then ripgrep reverts to the standard grep format (e.g., no line numbers). This means you should be able to use ripgrep in pipelines pretty much exactly like you would use grep. It just does the right thing. You can test this easily by comparing the output of `rg <pattern>` with `rg <pattern> | cat`. ag also does this to some extent (by disabling grouping and colors), but does still include line numbers in most cases.
> do you have an option like ag's undocumented -W which allows specifying a max line width for which a matching line is displayed?
Yes. That's the -M/--max-columns option. I have that set by default in my config:
$ cat ~/.ripgreprc
--max-columns=500
--colors=match:bg:0xff,0x7f,0x00
--colors=match:fg:white
--colors=line:none
--colors=line:fg:magenta
--colors=path:fg:green
--type-add=got:*_test.go
$ echo $RIPGREP_CONFIG_PATH
/home/andrew/.ripgreprcYes, I knew that. What I meant to say is that in the particular case of "rg pattern file", I feel it doesn't to the right thing. I understand that my usual usage of that kind of invocation may not be like the majority, so I understand that other people might prefer that default as it is right now. In my usual usage, though, I feel the numbers are burdensome, because I have to imagine the output without it to know what I'm passing in to the next command I've yet to type in the pipeline. I can't remember the last time I used that kind of invocation to look for the line number a match was in. I only ever use the line numbers when matching a directory or multiple files.
> Yes. That's the -M/--max-columns option. I have that set by default in my config:
That's awesome, and thanks for sharing your config.
That sounds both good (for direct use) and bad (for developing scripts) at the same time. And it's a general pattern. I wonder, is there a standard UNIX/Linux way of saying "run this, but pretend I'm not connected to a tty"?
I don't think this has been an issue for me yet, but it has tripped some people up, yes. Overall, I think it's worth it.
Or is because you deploy your scripts to other machines where ripgrep may not be present, but grep will?
* Deterministic
* Standard
* Written in c
* Equally performant (where it matters)
* Doesn't show colour or line numbers unless I tell it to
* Doesn't watch .gitignores unless I tell it to
(I should have said this in my first comment, but I'm the author of ripgrep, and I'm just generally interested in learning more about the differing use cases for these tools, in the words of users. I certainly have my own ideas about the question!)
% grep "die()" * [...results with functions named die()...]
% rg "die()" Error parsing regex
I reach for grep when I need to do non-regex searching. Can't remember how to do an fgrep fixed-pattern style ripgrep.
great tools andrew: I use them every day and install them first thing on a new box. know that your code really helps
For ripgrep, you can enable literal search the same way you do it in grep: with the -F flag.
Thanks for responding!
some | commands | grep ERROR
alias g=grep
EDIT: The real differences (at least the most important ones for me) between grep and these tools are 1) they format filesystem searches better for human viewing, 2) they provide more useful defaults, 3) in the case of ag, you can specify a max line width to avoid the terminal being filled by a matching line that has a ridiculous length, which typically happens with generated files, 4) they're supposed to be faster although I personally don't have searches big enough to notice the difference, but I imagine it's a big deal to others.
"Whenever bat detects a non-interactive terminal, it will fall back to printing the plain file contents."
The project is still rough around the edges but mostly good.
I'd like to recommend pygmentize also for syntax highlighting. Works like cat and supports a lot of languages. But since pygmentize is written in Python, it may not run as fast as bat. (I haven't tried bat though, because pygmentize is fast enough for my daily use cases.)
You can disable it with `bat --paging never`
I've been testing out a couple of different ones, like xd, fcd, wcd and pushd/popd, but I'm not quite sure which I should commit to or if there are better ways :)
7 │ alias ..="cd .."
8 │ alias ...="cd ../.."
9 │ alias gcd="cd (git rev-parse --show-toplevel)"
The third one will take you to the "git root" of a directory structure, i.e. the top-level folder where the .git directory is (I find this grounds me for when I'm doing git commands.)
You can even combine that with the aforementioned `fzf`: https://github.com/junegunn/fzf/wiki/examples#with-fasd-1
The readme also lists several alternatives in the same space as pazi.
The dirstack can also be saved to a file and reused on the next session.
Then I've stolen the "up" function somewhere to cd to a specific level using a part of the path when I'm deep in a FS tree.
Finally, not limited to cd only, Zsh can also expand paths with only the initials of the dirs, so 'cd /v/c/a/aTab' is expanded to 'cd /var/cache/apt/archives/', usually with less typing required on Bash...