Yazi: Fast terminal file manager based on async I/O | Hacker News Reader

Yazi: Fast terminal file manager based on async I/O

github.com

157 pointssbt56751 comments

Hi, I'm the author of Yazi, and I was pleasantly surprised to come across it while browsing Hacker News. I'm really excited!!

If you're interested in the topics of concurrency, performance, and io_uring discussed in this thread, you can take a look at my new article where I provide a detailed explanation of "Why is Yazi Fast?"

https://github.com/sxyazi/yazi/issues/143

This looks amazing! Just installed.

One thing if I may: I can't figure out how to use it. Like: The bottom right says "1 remaining", but I don't know what that is. I can't figure out how to run `rg` or `fd` from within Yazi.

Is there a user manual or a tutorial somewhere to read?

Thank you!

You can press `?` or `~` to open the help panel, which contains all supported keybindings.

This awesome project really could use better documentation / quick-start guide...

I took a long look at the "docs" but I did not find them helpful at all [1]

Even figuring out that you can press `?` to open the help panel took me way to long to figure out because it is not referenced in any obvious way.

[1] https://github.com/sxyazi/yazi/tree/main/config/docs

Yeah, it's possible that as a developer, I may not always promptly identify issues in the docs because the program tends to run "as usual" based on my ingrained thought processes...

I would be more than willing to stand on the user's side and continually improve the docs. If you would be interested in creating a quick guide for Yazi that would be great if it's possible :)

I was on the same situation and almost gave up using yazi, but I found a keymap configuration preset file here: https://github.com/sxyazi/yazi/blob/main/config/preset/keyma... that shows how to use most features and their respective key binding. Hope it helps!

This seems like very useful for browsing NFS shares. Has networked filesystems been a part of the initial motivation?

Hey there. Just wanted to say the code is really good in this.

It's super simple to read (once I found the setting to display tabs as 4 chars in GitHub), well organized, and has some neat patterns that seem to work well for building out TUI apps. Come say hi in the Ratatui discord sometime.

That's impressive. I wish the tabs could be shown side by side.

This is... wow. I really like this.

Do you plan to cache folder sizes?

I used Yazi for about a day and loved it, but there's just enough ergonomics missing that I ended up switching back to nnn for now.

My main issue I couldn't find a way to filter the list of files/directories for the current directory I'm in. Let's say I want to go to directory `cpp`, and it's 22 entries down the list of current files/folders. If I try to `fd` for `cpp`, I'm going to get back a ton of results from all over my hard drive. So it seems like I need to just scroll way down, which is a bummer. In nnn, you can just type `/` then `cpp` and it will filter results in CWD that match.

A nice-to-have would be live fuzzy-find functionality for the fd/rg integration, but that is not so critical.

The other thing I like about nnn is it retains `g` and `G` from vim to go to the top and bottom of the list of results respectively. I didn't see a similar operation in yazi.

Otherwise, very awesome tool, the performance is excellent, I appreciate how you explicitly state the external shell programs you interface with, I love the built in previews, and the devicons look clean.

Thank you for your appreciation!

In fact, `G` and `gg` are already supported in Yazi. You just need to set `arrow -999` for `gg`, and `arrow 999` for `G`. I've added it to the default keybindings: https://github.com/sxyazi/yazi/commit/c540542da49004a3084b1d...

The enhanced find/filter feature has been noted in the Feature Requests (https://github.com/sxyazi/yazi/issues/51), and I will make an effort to implement it soon. Of course, PRs welcome if possible :)

Is asyncio over engineering for this line of work? How much concurrency is really needed here and couldn't it have been addressed by threads instead?

But you see, there's a problem right there - threads aren't cool anymore. Not that the user knew or cared, but it's important perceived concurrency is shoved through the lens of async, somehow.

If you look at the video showing how it works, you will see, that it apparently starts displaying file content, while you navigate over files and directories. I think this is what happens asynchronously, in order to never block the ability to navigate.

I am not sure I am a fan of this, because I don't actually need everything to start being rendered. Maybe rendering something when I press a specific button would be good enough and that wouldn't necessarily require async stuff.

Personally I don't get the blowback that's shown up a couple places.

Why do we so readily convince ourselves against doing the right thing, picking good technology? "Over engineering" is a huge massive bat of a word that is applied to "correct" all kinds of spirited ambition.

Fear Uncertainty & Doubt is a dangerous default mode to adapt. I do not feel that it should be such an ordeal, require such courage, to do the good & right things. Better to try & fail than risk whiling ones time on lukewarm efforts.

What is the actual danger here? What are you afraid of? Why hold your biases against another's endeavor?

Async creates race conditions and even if you show me a proof that says there will never be a deadlock or other unexpected behavior I generally will believe that either there is an error in your proof or the actual executing code is different from the algorithm you've proved correct in some significant way.

Doing previews in a non-blocking way sounds really cool though, I'm not ragging on the concept, just saying I am mistrustful of async.

This is Rust, which is a language that doesn't have data races (though other kinds of races are certainly possible).

For an application like yazi async is likely a much more pleasurable experience to write code in than trying to manage threads and cancellations by hand.

Given async file IO in Rust without io_uring (and without being on Linux) means "dispatch work onto a background thread pool", it's essentially the same. It's just using a different programming model - which you might either prefer or not.

The effort expenditure is not the same, at all. Not by a long shot.

I've tried using LF in the past, but it didn't stick. Will definitely give this a go, as I'm trying to move to an pure terminal workflow as closely as possible.

https://github.com/gokcehan/lf

How does it compare to ranger, besides not being written in Python?

I think this has been listed at the head of the README, and includes a example video, take a look at it :)

It looks pretty cool! A nice feature of nnn is that you can set it up to `cd on quit`, meaning when you navigate to a different folder and then quit the program, your original shell navigates to that folder. Is that possible in Yazi?

The README includes are you looking for: https://github.com/sxyazi/yazi#usage

Gah. I need to skim better. Thanks. Here is a translation for anyone else who uses fish. Just hit it with a :%s/^\n//g to fix the formatting.

# Rename this file to match the name of the function

# e.g. ~/.config/fish/functions/ya.fish

# or, add the lines to the 'config.fish' file.

function ya

    set tmp (mktemp -t "yazi-cwd.XXXXX")

    yazi --cwd-file="$tmp"

    if set cwd (cat -- "$tmp") && [ -n "$cwd" ] && [ "$cwd" != "$PWD" ]

        cd -- "$cwd"

    end

    rm -f -- "$tmp"

end

Io_uring capable? That'd be cool.

"async Io" doesn't mean a ton to me, other than hopefully not blocking. It'd be cool to have a great well integrated terminal file manager that uses high performance io_uring.

That said, I have a hard time figuring out when & where I'd want to use terminal file managers. I like the idea, but the flexibility of the shell is so unparalleled.

How would the performance of io_uring would merit its complexity for a terminal file manager?

Wouldn't io_uring complexity be hidden behind the async io abstraction? Would the file manager even have to bother what underlying mechanism provides the async IO interface with the kernel? (epoll, kqueue, io_uring, etc)

Are there know ways that io_uring would leak through such an abstraction?

I was assuming the file manager would have to include the code for the async abstraction. But of course it could be just part of the async runtime / standard library. Still, a file manager doesn't seem like it would really need the ability to be so massively concurrent.

I'm also lost. Does it just mean that the UI stays responsive while a file copy happens in the background?

It says "CPU tasks are spread across multiple threads", but what CPU tasks does a file manager perform?

Almost every UI framework out there is single-threaded and has facilities to help offloading such tasks into the background. Usually, to a worker thread pool. It's a bit fiddly, especially in a non-managed language that doesn't protect from dumb mistakes, but it can be easily done.

io_uring is for use cases where the context switch overhead of system calls starts to matter. While async I/O can be justified for U/I programming, it's safe to say that a file manager won't ever benefit from io_uring.

There is a Reddit post discussing this point: https://www.reddit.com/r/rust/comments/16fxr58/comment/k04u7...

> There are many scenarios that require CPU, such as image downscaling, image encoding, code highlighting, sorting large directories, etc.

If you want to delve further into Yazi's inner workings, you can see https://github.com/sxyazi/yazi/issues/143

What advantage does thread pool + io_uring have over just thread pool?

Unless you have millions of CPU cores and therefore many downscaling/encoding/highlighting tasks starting/finishing every second, I still don't see what io_uring brings to the table.

Running tasks in the background does NOT require io_uring.

Hey, Yazi currently is NOT using io_uring, and the link provided above explains it thoroughly, please take a careful look.

The reason I use a term file manager is because it makes some cumbersome things much smoother. With nnn I can navigate directories at least as quickly as with a mouse in a GUI, can create new directories/files by pressing just two keys, and batch rename selected files in a Vim textbuffer. Also it's useful as file browser in Vim.

I don't use it for everything and I still copy/move files a lot without it. But it accelerates some things and is simple to use.

Hi, based on the current architecture of Yazi, I don't think it would be difficult to add io_uring to it.

This is something we can consider for our future work, as it's not the current performance bottleneck.

For more details: https://github.com/sxyazi/yazi/issues/143

Nice write-up! It shows that you didn't just use async for the sake of it, but that you thought about the kind of user experience that you want to have, and how async helps you to get there.

How does it compare to nnn?

Awesome project. Will try it. I can see how it can be useful and fun to use it in remote ssh sessions. When DOS was a thing I just used Norton Commander. But with linux the state of existing file managers made me get used to working in plain command line. Yazi seems like a stride in a good direction.

After using it for brief while, I'd say that I like it. It will not replace may daily driver (Midnight Commander), but it's good for fast preview/rename/delete actions.

In 1991 dos navigator had an integrated file manager and editor like no other console file manager has: you could alt-tab and copy/paste from editors into shell commands interactively.

In 2023 I still need two extra terminals for vim and for bash. And text selection is mouse-only.

Doesn't Emacs fit that bill? In Emacs you can run terminal emulators and various shells and from all of them you can copy and paste into other buffers or from other buffers into them.

To add to this: I had lots of problems with Emacs terminal emulators until I switched to VTerm. Most things work like you’d expect in VTerm.

Yep, vterm is great. Only thing annoying me is, that I have to switch modes to copy anything. Wish it would work like `M-x shell` instead, where I can simply navigate upwards and when I move point back to the prompt, I can paste and continue with commands.

Vim has split screen support and built-in virtual terminal supporting normal mode. For file manager can use fern written in pure vimscript. In 2023 you were just not aware of them.

>> In 2023 I still need two extra terminals for vim and for bash. And text selection is mouse-only.

Terminal multiplexers like tmux, GNU screen, and others allow for keyboard-based text selection and copy / paste.

Many advanced editors like VS Code, Emacs, and other have built-in terminals and keyboard commands to select, copy, and paste text.

Still, DN experience beats everything what we have today. Of course there were lot less features and many limitatitons, but the overall experience was great.

tmux does this too if you use buffers. Check out zellij. If using a multiplexer is not for you and you're all in a graphical environment, I believe you might find kitty worth a closer look.

> In 2023 I still need two extra terminals for vim and for bash.

wat

Vim and Emacs have advanced navigation modes that can totally make up for lacking mouse support.

On the console, many use cases where you'd need copy&paste can be covered by autocompleters. For example branch names in version control. Someday I'll have to look into how to write my own...

Edit: with !, you can execute a shell command and watch its output without leaving Vim.

Reply on news.ycombinator.com