Hacker News Reader

Comment by pclmulqdq | Hacker News Reader

Text files are very hard to efficiently parse in parallel because there are so many variable-length and conditionally present fields. That necessarily leads to branches in your code which GPUs are not designed for.

View on HN

It depends on the format of the text, the parser for that format, the language of the parser, and many other factors. Such a broad affirmative statement that text files are inefficient is tenuous at best.

We need some technology that turns text files into binary. To the startupmobile!

Hmm sounds like a compiler

Then pad fields and require them, even if the value is Null/None/Nil, and the field names too.

  USERNAME:dotancohen\0\0\0\0\0\0
  FOOBAR\0\0:\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0

This might look ugly if you open it in Notepad, but so do XML and Org mode files. However an Emacs/VIM/VsCode plugin could make this seamless for the user.

They might not look as pleasant as Markdown, but Org mode files still look good in a plain text editor like Notepad.

Reply on news.ycombinator.com