leonim

This is the = command that adds a new column based on a Python expression you provide. You can reference the cells in the current row based on the column name.

It is also possible to create a column based on the output of a shell command.

VisiData is row based, not cell based, so you cannot create expressions that reference data across rows.

Try asking on https://github.com/saulpw/visidata the community there can help.

I am seeing this issue on a Mac with ghostty but not with wezterm, from the same binary. Ghostty shows some correct characters and some diamond-question marks, and a line usually contains one or the other. The ghostty inspector shows the diamonds are not the blocks.

Thank you for doing this, this is fun. Also, I love all the nostalgic comments.

I'm surprised no one has mentioned Robert Tinney the artist who illustrated many of the covers. His work is pretty amazing, and seemed to capture all sorts of interesting ideas.

I met him once at a computer show. I was in too much in awe that I didn't really talk with him, but was able to give him some praise for his artwork. Also, I bought one of the t-shirts with his artwork.

One my favorite tools is Visidata (https://www.visidata.org/). It is great for viewing and working with data in the terminal. It is useful if you need to quickly explore data and change some data.

VisiData makes it possible to view a lot of data such as CSV, JSON, databases, etc. All the supported formats are listed in https://www.visidata.org/docs/formats/

There is a wonderful tutorial in https://jsvine.github.io/intro-to-visidata/

One of my favorite tools. However, I don’t think that Visidata is a spreadsheet, even though it looks like one and is named after one. It is more spreadsheet adjacent. It is focused on row-based and column-based operations. It doesn’t support arbitrary inter-cell operation(s), like you get in Excel-like spreadsheets. It is great for “Tidy Data’, where each row represents a coherent set of information about an object or observation. This is very much like Awk, or other pipeline tools which are also line/row oriented.

For CLI tools, I’m also a big fan of Miller (https://github.com/johnkerl/miller) to filter/modify CSV and other data sources.

This works great with terminals that can handle ligatures. I've used it with Kitty and recently tried it with Ghostty. The author shows several different examples using the fonts to make groups of 3 digits within larger numbers easier to read by adding underlines, grouping numbers closer, bolding numbers, and adding virtual commas. I say the commas are virtual since they are only visible by the font rendering, if you were to copy and paste the numbers the commas would not be included.

The author provides a font patching tool, however mentions that it does not preserve existing ligatures.

This font can be useful when looking at large numbers. I've used it with successfully with Visidata in sheets with large numbers.

If you are interested in this subject, Carl Zimmer wrote a great book that has all sorts of examples of parasites that control their host: "Parasite Rex: Inside the Bizarre World of Nature's Most Dangerous Creatures"

https://www.amazon.com/Parasite-Rex-Bizarre-Dangerous-Creatu...

Atul Gawande is a surgeon by training and an exceptional writer. He has authored several other wonderful books, known for his ability to explain complex topics in an accessible way. His website (http://atulgawande.com) features his books, articles, and papers along with information about his other endeavors.

I hope he'll return to writing.

There was a lot of effort to add many more help guides that provide context-sensitive help where you need it. The main complaint I’ve read from people is that VisiData is too complex. The last few releases have attempted to address these concerns with the guides, the command palette, and the menu system. I think these all provide help when it is needed.

Though I think it is possible to get a lot done with VisiData only knowing a handful of commands/operations.

I think that the Jeremy Singer-Vine tutorial is a great starting point: https://jsvine.github.io/intro-to-visidata/

Also, the VisiData developer Saul Pwanson has a YouTube playlist listing all sorts of VisiData demos and step-by-step videos:

https://youtube.com/playlist?list=PLxu7QdBkC7drrAGfYzatPGVHI...

This is cool.

I'll mention an alternative way to visualize Venn diagrams are UpSet Plots.

[1]: https://en.wikipedia.org/wiki/UpSet_plot [2]: https://upset.app/ -- points to multiple implementations.

I would add another site to the list: https://inconsolation.wordpress.com/

You can find reviews of hundreds of CLI And TUI programs, some great, others barely known and clearly forgotten. The last review is from 2015. Some reviews are very short, others are more involved.

The site is a blog with a tagline "Adventures with lightweight and minimalist software for Linux". The author K.Mandla is opinionated and has certain preferences for such programs like [1].

I have not looked at the site in years, but if you are willing to explore, this could be a fun option to find a variety of these tools.

[1]: https://inconsolation.wordpress.com/2015/01/19/wpe-and-we-so...

I don't think this would work as a lesspipe.sh viewer, since it is interactive. I have wanted this for VisiData for data files. You would need a wrapper around less that could start other programs instead of less.

You might be interested in Tidy Viewer with lesspipe.sh

[1] https://news.ycombinator.com/item?id=28670252 [2] https://github.com/alexhallam/tv

This looks really nice, I hope it does well.

It reminds me that I miss https://inconsolation.wordpress.com/ ended. It had all sorts of great and humorous reviews on terminal based software.

Nice to see progress on Oils. The blog post talks about some screencasts, does anyone know where those are?

This fx rewrite is very exciting. I'll have to try it. I thought of fx as a wrapper around jq, that allowed quick iteration over building jq scripts. Sort of an Ultimate Plumber [1] but only for jq. It looks like it is now more like a JavaScript processor plus an interactive viewer.

Someone mention Visidata[2]? VisiData is also a TUI that is great on tabular data, and it can work with json. If your JSON is mostly tabular in nature, Visidata does a great job at showing that data and allowing you to explore it. A lot of json I deal with is tabular-like data. There is a great tutorial [3], that can help you get your bearings with Visidata. Once you understand those basics you might want to look at this thread [4] for what commands you can use with json.

[1] Ultimate Plumber: https://github.com/akavel/up

[2] Visidata: https://github.com/saulpw/visidata

[3] "Intro to VisiData Tutorial": https://jsvine.github.io/intro-to-visidata/ by Jeremy Singer-Vine

[4] "Is it possible to "flatten" structured data (like JSON?)": https://github.com/saulpw/visidata/discussions/1605

I also find VisiData is useful for adhoc exploring of JSON data. You can also use it to explore multiple other formats. I find it really helpful, plus it gives that little burst of adrenaline from its responsive TUI, similar to fx and jless mentioned.

For my toolbox I include jq, gron, miller, VisiData, in addition to classics like sed, awk, and perl.

- https://github.com/saulpw/visidata - http://visidata.org/

Also there is a great introduction: - https://jsvine.github.io/intro-to-visidata/ "Intro to VisiData Tutorial" by Jeremy Singer-Vine

I would be interested to see what an LLM could make out of this archive. How it might categorize the threads or how it might help make sense of it. Vikram's manual curation is the useful part. In the podcast, he mentions he would like to see what could be done with an AI and the archive.

Actually the Vikram said he had initially added a search feature and it didn't appear to be very useful. He says:

   search is an awful way to curate text when your readers don't know what is interesting about it. This is obvious in retrospect, but it didn't occur to me until I showed what I built to folks.

Though there are at least 41 different tags that he applied to the threads and can be found here: https://atariemailarchive.org/categories

He talks about this in his post: https://vikramoberoi.com/how-i-made-atariemailarchive-org/

I originally heard about this on the Data Is Plural podcast: https://podcast.data-is-plural.com/2159594/12535597

The e-mail archive of one Atari Engineer, Jed Margolin, who worked in the CoinOp division of Atari. Jed was a hardware engineer at Atari from 1979 to 1992, and these messages comprise his inbox while he was at Atari.

With permission, the owner of this site, Vikram Oberoi, curates the archive, including pointing at interesting threads. There was no threading in the VAX email system they used, Vikam links together the threads.

Neither Jobs or Wozniak are mentioned, they left before these threads. But there are interesting topics like layoffs, piracy, residuals, pop culture, babies, etc.

(Those babies should be into their 30s now. Wonder if any are reading here.)

The VisiData developers created a fun multi-part data wrangling puzzle last year called the Hanukkah of Data [1].

It is not specific to any tool, so you can use command line tools, the TUI data explorer VisiData[2] or whatever data tools you want to solve the puzzle in your terminal.

[1] https://hanukkah.bluebird.sh/5783/ [2] https://github.com/saulpw/visidata

Jeremy Springer-Vine, the Data Editor, BuzzFeed News talks about the data efforts that underpinned the FinCEN Files.

I wonder if this was today, if they would have used AI data-scraping to get structured data out of the PDFs.

lolive, VisiData has some Excel support. However, don't expect VisiData to be a full blown editor for Excel files. It can provide a view of the data in an Excel spreadsheet.

Also, there is the VisiData Tutorial by Jeremy Singer-Vine. This is a very helpful introduction to VisiData.

https://jsvine.github.io/intro-to-visidata/index.html

I am a big fan of VisiData for working with data in the terminal. I also like John Kerl's miller[1] and Simon Willison's sqlite-utils [2] when I want to do queries. These are useful for working with data in simple shell scripts.

[1] https://github.com/johnkerl/miller [2] https://github.com/simonw/sqlite-utils

Yes, agreed. This looks like it is being actively developed. It has some interesting ideas. I like the fact that conflicts are committed, and can be resolved later:

"""If an operation results in conflicts, information about those conflicts will be recorded in the commit(s). The operation will succeed. You can then resolve the conflicts later. One consequence of this design is that there's no need to continue interrupted operations."""

I like Perforce it is fairly easy to use.

If you want to go old school there is RCS or SCCS. GNU provides source code (for SCCS there called CSSC). Though *CS are per file not per logical commit.

IIRC Perforce actually used RCS under the covers for storing the deltas.

I remember getting a pirated copy of this game many years ago. A friend at school would give me bootlegged disks and this was one of the games. I would just get the disk, and no instructions. I think I played it a little, but didn't get into it. I didn't really understand how to play. Maybe I had seen a review in a magazine with some details.

My Dad was an economist. When he saw this game, he was really impressed by the interface. He thought it had some really interesting ideas, like the auction mode. If I remember correctly, he thought it could help as a teaching aid.

Nice to see it back. I'll have to try it again.

Is there a TL;DR about the current state of the Oil Shell. I'm aware of Oil Shell and hope it succeeds, but have not been following over the last few years.

VisiData learns some new tricks that allow it to generate SQL queries directly, and explore the results all in the terminal. The author gives a demo of the new tricks.

To me it is amazing at exploring data. I wish we had that same interface on every place I'm looking at a table of data.

I can't see it replacing Excel or other spreadsheets for people who make heavy use of spreadsheets in their jobs. For a terminal junkie like me, who wants/needs to make sense of data that is much bigger than my screen, Visidata is much better than a spreadsheet. I can filter and partition data easily.

I used to have to use perl/awk/sed/sort/comm/grep to help me put that data in some format that was useful. Visidata is much easier and faster to use.

Here are Saul Pwanson's YouTube videos showing off Visidata. Saul is the developer of Visidata:

https://www.youtube.com/playlist?list=PLxu7QdBkC7drrAGfYzatP...