Lsd: The next gen ls command
github.com
github.com
[1] https://github.com/ryanoasis/nerd-fonts
[2] https://github.com/ryanoasis/nerd-fonts/wiki/Codepoint-Confl...
https://github.com/saitoha/libsixel
By high-resolution I mean you can display JPEGs using sixels. You can run OpenGL programs using sixels.
Anyway, one nice thing about TrueType/OpenType fonts is that the vector representation is extremely well compressed. SVG or similar would be an efficiency loss. In this light, fonts are actually a fairly decent and practical way to represent vector symbols that will be cached and reused. Maybe a minimal practical solution might be to add an ANSI extension that requests an appropriate font for a particular area of the screen, perhaps with a URL from which a specific webfont can be fetched.
I agree that an external resource is a lot of hurdles. What I had in mind while writing the reply was a stripped-down version of TrueType, with only a single `glyf` table allowed (probably with a minimal version of the `head` table as well). Judging from how icon fonts are used today, this might even be possible without a persistent font state.
stty saneAgreed, and I don't understand why most of the things still done in the Unix terminal have not transitioned to newer, less hacky environments.
I managed to replace the vast majority of the things I was using the terminal for with uses of about 400 lines of Emacs Lisp code that I wrote about 5 years ago.
A large fraction of my uses of the terminal since then have been for the purpose of my getting my Emacs environment and my data files installed on a fresh install of my OS and for recovering from those times when I inadvertently introduced a bug into my Emacs environment that my regression tests did not catch.
The 400 lines of Emacs Lisp code submits a string I typed to bash with the -c flag then asynchronously (i.e., without blocking Emacs's UI) waits for output from the bash process. I also added a way for the user to arrange for a bell to ring when the process finishes. Notably, there is no provision for the user to cause anything to be sent to the process's stdin after the process is started, which allows the 400 lines of code to use a pipe instead of a pty (which is relevant to this comment thread because if OSes didn't need to support the pty abstraction, they could be significantly simpler). In the rare case when I want to send a string to the stdin of some process I use the shell to set things up:
echo "here is the string" | some process
That whole command line becomes one argument to a call to bash. The 400 lines of code I wrote cannot be used to interact with a REPL. E.g., if you use my 400 lines of code to run "python" by itself as a command line, the python process waits for user input, which will never come because there is no way to write to the python process's stdin.The closest analog in standard Gnu Emacs to the 400 lines I wrote is "shell mode" (shell.el) which consists of about 3 or 4 thousand lines of Emacs Lisp code. My code re-uses shell mode's code for interpreting (ignoring in my case) the escape sequences for ANSI color and for interpreting the escape sequences for "carriage control", but (like shell mode) does not handle the escape sequences for cursor addressing. IIRC the main reason I was dissatisfied with shell mode was that it contains hacky code to keep the bash process's "opinion" on what the current working directory is in sync with the shell mode buffer's "opinion" on what it is. In contrast, in my code, the buffer's "opinion" is the only one that matters (because every command line is processed by a fresh bash process on that bash process inherits its current working directory from the Emacs process when it is created).
If I ever were to start to need to issue a great many command lines on a variety of remote Unix servers, I would probably take the time to write a replacement for ssh along lines similar to what I just described.
It would not be a trivial amount of work to build this since, to take good advantage of the UI capabilities, you'd need an entire new CLI OS - something to replace GNU Core Utilities. I haven't been sure where to start since the choice of programming language or expression language will significantly affect the overall experience.
OTOH, if you do what your parent suggests and start from scratch (no terminal support) you have a platform that isn't supported by anything yet, so unless you port or reimplement everything you need you have to keep using both in parallel which sounds just cumbersome.
Honestly, it's likely that nothing special is needed on the ssh part of things as simple port forwarding and environment variables would do the trick.
If the target audience for this code were users other than just me, then it might not pay to arrange things that way, but all the motivation I personally needed to remember to treat the cd command differently while using the code I wrote was the very pleasant thought that my giving up on maintaining normal behavior of the cd command means that for my usage patterns, which again do not including using the code I wrote to interact with REPLs[1], no state need be maintained by any code whatsoever outside of the Emacs process except for the usual, expected state of the file system and any state internal to any bash processes and any of their child processes that are currently executing a command line.
Standard Gnu Emacs contains a M-x cd command, and that does modify the wd in a way that persists through the issuance of multiple command lines. (Again, I arranged for Emacs alone to keep track of the wd.)
[1]Standard Gnu Emacs contains the command M-x run-scheme, M-x run-python, etc, for interacting with a Scheme REPL, a Python REPL, etc. I do not happen to use any of them, but if I did start to use one of them, I probably would not feel tempted to re-write it the way I re-wrote shell mode (a.k.a. M-x shell) because I can think of no easy way to arrange things so that no state persists inside the REPL between command lines. What is special about interactive shells is that there is this single dinky piece of state, the wd, that if treated differently, makes the interface between the terminal or terminal-like thing and the shell process "connectionless" in the way that HTTP is connectionless: namely, after a command line finishes, the shell process and the terminal-like thing can just forget about each other.
I would imagine that this arrangement would be even more worthwhile for running shell command lines on lots of remote machines, but haven't written any code to effect it because I personally don't do that often.
ADDED. If I found myself writing "VAR=3 foo command" more than twice I would probably arrange for "VAR=3" to be permanently added the appropriate emacs variable (namely, process-environment IIRC). Basically, in my life the only environment variables that matter can be set once in a config file somewhere and forgotten.
[1] https://www.unicode.org/L2/L2019/19068-powerline-syms.pdf
This usually works not by having fonts define emojis (most don't) - it's just that systems reach in and render emojis from a dedicated emoji-defining font when they encounter them. So it's not like the world is full of fonts that define 'standard emojis' or funky powerline symbols.
The only more or less widely used solutions are iTerms image support (PNG images, no vector), or Sixel (xterm and some other exotic ones).
There is also ReGIS (https://en.wikipedia.org/wiki/ReGIS), which is a vector format, which might be supported by xterm (not sure).
For raster images there are sixels, and for vector there are the tektronix codes. These are standard in xterm since forever.
Of course, they could be modernized, using real raster pixels and something like a subset of svg. I guess before standarization it would be better to have a reference implementation included in a major standard terminal.
A standard xterm supports vt340 if you start it with the option "-ti vt340" which supports ReGIS as well.
Now I feel like all my hard earned cli-fu is a lie.
Anyways, Id been away from *nix for a few years at the time, so I was a bit rusty and soon felt like a moron and closed the bug once I realized my error.
> Now I feel like all my hard earned cli-fu is a lie.
fd is also a lie. If you want a decent interface to find just use "find | grep ..."
Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore.
See the guide for more details: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a...
> Performance difference is not that much of a dealbreaker in practice
This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file:
$ time rg '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
3003
real 1.608
user 1.105
sys 0.496
maxmem 9473 MB
faults 0
$ time ag '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
ERR: Skipping OpenSubtitles2016.raw.en: pcre_exec() can't handle files larger than 2147483647 bytes.
real 0.016
user 0.004
sys 0.011
maxmem 6 MB
faults 0What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files.
If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve.
One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping.
So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day.
BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).
As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work.
You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.
Oh, thanks, I've missed it somehow.
> rg foo | less -F
Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn }
It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does.
EDIT: actually found a bug report that seems to describe exactly the same problem. https://github.com/BurntSushi/ripgrep/issues/513
If you're willing, I'd definitely appreciate it if you could update #513 with your environment. In particular, by answering these questions: https://github.com/BurntSushi/ripgrep/issues/513#issuecommen...
If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great.
Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.
Show results with every match on its own line, including line numbers and column numbers. With this option, a line with more than one match will be printed more than once.
$ echo foofoo | rg foo --vimgrep
<stdin>:1:1:foofoo
<stdin>:1:4:foofoo
What you want is probably `--no-heading`. You can add it to a config file or an alias if you always want it enabled: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#c...I'm sure you already know this, but sed is meant to provide the features that ed does, so sd is quite a bit more limited. Still, I can see how this can be useful, as most invocations of sed end up only really using a handful of its features.
Although personally, I've only really used sed for find & replace so sd solved my gripes with sed perfectly.
I guess my brain just refuses to remember these for some reason.
However, here's my reasoning for the first of your questions, as a happy fd user who switched from GNU find. Maybe it can give you insight into why the UX is designed as it is.
The extension is distinguished from the filename because it's a common use case (-e jpg less keystrokes than -iname '*.jpg'), it's semantically separate from the filename (IMO) and therefore should not be treated as part of it. And unlike filenames, extensions most often do not need to be pattern matched. If you want to match multiple extensions, just repeat the -e flag e.g. -e jpg -e png.
If you feel that there is anything we could to to improve fd's UX, it would be great if you could share it on GitHub: https://github.com/sharkdp/fd
> Why separating the extension from the filename? Is this a default
It does not need to be! You can just also just use "fd README.md". Admittedly, the "." will actually search for any character (regex), so if you want to be precise, you need to escape it.
Would love to see this resolved, cause it seems rather nice otherwise.
So please drop the dismissive patronizing tone.
It’s obviously not a matter of needing, but of aesthetics and usability, and that’s very much a valid pursuit.
How does dircolors add support to nerdfont?
How does it add different colors to user, group, permissions in the equivalent of ls -l?
Also, dircolors is not included in OS X.
I probably run ls a few dozen times a day, but it's usually just to answer "what did I call that file again?" or "did I remember to copy that over?"
I can't imagine a scenario where I'd require too many extra visual cues. I use color ls, mostly because I find the optional symbols like '/' to be distracting, and differentiating files and directories via color is about all I care about.
There's overhead in becoming too attached to non-standard tools: you have to go out of your way to install them, your muscle memory needs retraining, etc.
Most importantly, tools like this break a bunch of core command line standards, like doing one thing and doing it well, and producing clean output that can be pipeline-filtered with all the other Unix standards.
Playing the "doing one thing" card when it comes to ls is rather risky, as ls, with its options for doing things like sorting its output, is one of the favourite examples that people point to of tools that do not adhere to this philosophy in practice.
* https://news.ycombinator.com/item?id=4023265
* https://news.ycombinator.com/item?id=9673975
* https://news.ycombinator.com/item?id=8484718
... and so on.
https://github.com/alebcay/awesome-shell
Only for rust but not a bad list I think.
https://lib.rs/command-line-utilities
Some blog post introducing cli tools.
1. People predict Rust will be more popular in the future, so they're trying to learn/master the language by practicing it.
2. People think speed and memory safety are important guarantees for unix programs.
3. People are testing whether Rust deserves the fame it has
I don't think any of the typical unix tools ever segfaulted on me in the last 15 years I use Linux.
Something as simple as `ls` is probably so small and battle tested that it's not an issue, but if you're writing an all-new, not-battle-tested tool, why wouldn't you want stronger guarantees? The new tool is being written for the features, but it's not fun to write vulnerabilities into something that should be simple and "just work."
Languages like Go and Ruby are also mostly memory safe, so those are generally fine picks here too, but every language has trade offs. In this case, the author clearly cares about performance, which Ruby does not care about.
Rust also has a built-in testing harness, which is a lot more convenient IMHO than using whatever C testing framework you might have a predisposition towards.
std::string foo(bool flag)
{
if (flag)
return "true";
}
Questions I'd ask:
* Is the function well formed (Yes - functions need not return a value on all paths due to C ancestory, even if the return type has a non trivial actor - not sure if this is still considered well formed, but I think it was as least to C++11)
* what happens if 'foo' is called with true? (Returns "true" as one would expect)
* what happens if 'foo' is called with false? (Undefined or implementation defined behavior, but generally nothing nice - segfault, acces violation, etc)* if it crashes, where, when and why does it crash? (Technically since its undefined, nearly any answer here suffices, if it can be backed up. Since practically, most optimizing compilers assume UB can never happen, when you return nothing from a non-void function, the compiler will attempt to invoke the destructor of a non existent object instance (assuming non POD) and boom.)
I asked this because it was a distilled example of a real world rare crash that was extremely difficult bug to track down because the crash location is often know where near the offending function.
I remember getting into a heated argument with a coworker when I claimed it should have been a compilation error. IIRC, he claimed it to be a halting problem and that the compiler couldn't determine that all paths didnt return a value. I called BS, citing at the time (circa 2004) that the new compiler on the block for C# could reliably emit errors when not all return paths returned a value.
I also like it that in C++, it's a very rare example of a very terse example dealing with a number of topics such as undefined/implementation defined behavior, debugging, compilation settings (warning levels, etc) all in a mere 4 lines of codes. With 4 LOC, which is straight forward and simple for the candidate to mentally parse, I can gleam a lot about their understanding of the language (and it's potential pitfalls).
Sorry if this got a little long winded and ranting.
Your friend was confused because you cannot decide whether a certain line in your program will be executed. Since this is equivalent to the halting problem. Let's prove this! Assume we have a blackbox B(P,x,N) decides whether line N of program P will be executed given input x. Here, we can solve the halting problem:
```pseudocode
def Helper(sourcecode P, input x):
[0] P(x)
[1] print "What Do We Say to the God of Death?"
def Halts(sourcecode P, input x): if B(Helper,(P,x),1):
return true
else:
return false
```This means, given programs like your interview question, compiler cannot decide -- in general -- whether the program will crash or work. Of course, it's not too hard to find "good enough" heuristics that'll catch some cases and/or restricting your language to make it "harder" to encode such programs.
In C falling off the end of a function without returning is legal, as long as you don't use the return value. In C++ this is illegal.
Theoretically we can't determine whether a function will return a value or not. In practice, heuristics get 99% of the way and the last 1% you can make the programmer put in a possibly redundant return statement.
Answering the second in the general case is equivalent to solving the halting problem. But answering the first question is much simpler. Static analyzers aren't using a heuristic approach to the second question, they're solving a completely different problem.
Rust is great in this sense because you don't have to think about those issues.
That said, I'd pick rust over C for most projects simply because it's a nicer language to write IMHO. It feels like a modern high-level language with things like iterators, traits, etc, and it has a great package manager too. In comparison, I find writing C to be a huge chore, especially if you're doing much string processing
Rust will not help you with other multi-threadding issues, that arise from how OS implements processes/threads/signals/syscalls etc.
So I wouldn't call it fearless. Concurrent data access is just one issue you have to tak care around.
That library is used by most(?) rust cli tools, so they all have a similar feel. In addition, there is a library built on top of clap called struct-opt that makes bringing up a new cli exceedingly easy: https://docs.rs/structopt/0.2.15/structopt/
Seriously, checkout struct-opts page even if you don't know any rust. I think their demonstration on the front page summarizes a lot of things I like about idiomatic rust.
- Compiled so easy to distribute compared to Python, JS, etc - Cargo makes it easy to pull in libraries compared to C/C++ - Smart people made some great libraries that make it trivial to get performance and/or a nice UX. e.g. Everything in ripgrep is available for reuse, down to how it walks the filesystem.
It’s also a lot faster, according to the benchmarks.
A modern replacement for ls, written in Rust. https://the.exa.website/
ls with coloring and icons https://github.com/Electrux/ls_extended
However it's written in our lord and savior RUST and has lots of colors so this shit is definitely "next gen."
0: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls...
So you're using this as an opportunity to bash an entire language ecosystem? The merits of this project have no bearing on Rust or its adoption.
I think the parent was bashing the attendant hype and the resulting reinventions but worse.
There was a blog I read about the Rust Zealot Brigade (or similar name) that would reliably descend on any HN conversation.
1. It’s written in the author’s preferred language