Hacker News Reader

Comment by nayuki | Hacker News Reader

I greatly prefer TSV over CSV. https://en.wikipedia.org/wiki/Tab-separated_values

100% agree. TSV is under-rated. Tabs don't naturally occur in data nearly as often as commas so tabs are a great delimiter. Copy paste into Excel also works much better with tabs.

Code editors may convert tabs to spaces but are you really editing and saving TSV data files in your code editor?

TSV's big advantage is that, as commonly implemented, the separators are escaped, not quoted. This means that a literal newline (ASCII 0x0A) is always a record separator and a literal tab (ASCII 0x09) is always a field separator. This is the format many databases -- including Postgres -- use for text export by default.

There are some notes I put together about TSV a few years ago that expand on these points: https://github.com/solidsnack/tsv?tab=readme-ov-file#motivat...

Unix and AWK will parse it on the spot.

The problem with using TSV is different user configuration. For ex. if I use vim then Tab might indeed be a '\t' character but in TextEdit on Mac it might be something different, so editing the file in different programs can yield different formatting. While ',' is a universal char present on all keyboards and formatted in a single way

Csv suffers from similar encoding and platform specific gotchas surrounding newlines and delimiter escaping.

Well, back in my day, we used tabs for tabs.

But then some folks came along about 15 years ago screaming about spaces, and they won, so now tabs are 2 or 4 spaces.

The law of unintended consequences strikes again!

Note: not meant to denigrate you space supporters out there.

The only situation I can think of where a tab is not a tab, is in a code editor that's been configured (possibly by default) to use spaces instead. But that's an easy enough configuration to change. And certainly wouldn't be a problem for something like TextEdit.

CSV is a pseudo-standard anyway, IMO the delimiter should be a configurable option (like it is in Unix cut and those kinds of tools).

> IMO the delimiter should be a configurable option

It is. CSV has been character separated vs comma separated for probably decades now. Most tools you'd use to mess with them allow you to specify which separator character is being used.

I agree although it seems to render the distinction made by GP moot, right?

Functionally the same. I'd prefer CSV if my content was likely to have whitespace in it and didn't want billion quotes. I'd prefer TSV if my content was unlikely to have whitespace, and more likely to contain commas.

The problem with TSV is what are you going to do about quotes. Some fields might contain them [1] or they might be needed to store fields with tabs inside them.

Because of this in order to read a plain simple TSV (fields separated by tabs, nothing more) with the Python csv module [2] you need to set the quote character to an improbable value, say € (using the euro sign because HN won't let me use U+1F40D), or just parse it by hand, e.g. row.split('\t').

[1]: https://github.com/wireservice/csvkit/issues/1194

[2]: https://docs.python.org/3/library/csv.html

Quotes are just characters in TSV. Actual tab characters are banned so this should be simple. (Of course poor implementations may behave differently.)

Agreed, much easier to work with, especially if you can guarantee no embedded tabs or newlines. Otherwise you end up with backslash escaping, but that's still usually easier than quotes.

It's not really easier than CSV if you can guarantee no commas or newlines.

Much easier to require fields not have tabs than not have commas, though.

Parsing escapes is easier than parsing quoted text with field and record separators embedded in it. Every literal newline or literal tab is a separator. One can jump to the thousandth record, for example, just by skipping lines, without looking into the contents.

TSV looks incredibly ugly when opened in a text editor unless all values are 7 characters or less.

Set a bigger tab size in the editor, e.g. `:set ts=32` in vim.

Aaaand now I can fit like 3 fields next to each other before they wrap and the field with arbitrary length text still misaligns the field after it

There is at least a chance that the text fields will fit in a tab. If not, a tool like “column” on Linux can be used. There is no chance that a text field will fit inside the width of a comma.

I was about to mention that column [1] is part of util-linux, so perhaps it's Linux specific, but then I noticed this in the FreeBSD man page [2]:

> The column command appeared in 4.3BSD-Reno.

[1]: https://man7.org/linux/man-pages/man1/column.1.html

[2]: https://man.freebsd.org/cgi/man.cgi?query=column&sektion=1

vartabstop can help with per-column width. I found

https://sharats.me/posts/automating-the-vim-workplace-3/#usi...

which greatly improved my TSV editing experience in vim. Couple that with autocmd TextChangedI and you get realtime recalculation and alignment.

There are some specialized text editors for editing files with tabs. https://en.wikipedia.org/wiki/Tab_stop#Dynamic_tab_stops , https://nick-gravgaard.com/elastic-tabstops/ , https://tibleiz.net/code-browser/

If have to use a dedicated tabular data editing program, I may as well use a spreadsheet application. What do the options you propose do better than libreoffice calc?

    column -t | less -S

is pretty nice because you can inspect a file or dataset on a server (no X) before downloading, to see if you even want to bother. Or you can pass it along through a series of pipes to just get the rows you want.

What part of "the value of CSV is to have something that can be easily viewed and modified in any text editor" do you not understand

I’m not sure where that quote is from but it is incorrect, CSVs aren’t easily viewed or modified in text editors in general (at least not in any way that takes advantage of their tabular nature).

There’s at least a slight chance that tab separated values will look ok in a text editor (although in general, nope).

How does CSV look when the fields are all of different widths?

Like a list of lists of strings

meh. The whitespace makes it easier to eyeball IMHO.

Thanks for your input.

Reply on news.ycombinator.com