Heres the .q file I use to read HN. Hobbyist programmer; corrections/improvements welcome. Better yet, how would I do this in J?

cat 1.q

   \c 2000 200
   k)-23!t:select from .:`:t.kdb
   k).Q.fs[{`t insert +:`url`title`user`item!("SSSS";",")0:x }] `:hn.csv 
   k)-23!t:?:select from t 
   `:t.kdb set t 
   \\

As for how to get hn.csv, I use shell scripts to download all the pages of HN html (in accordance with robots.txt specified delay). Then I use a simple lexer (made with flex) to transform the html to hn.csv with the above selected columns.

After some years, the file hn.csv can grow quite large, larger than available memory.

To read and select articles from t.kdb, I use less(1) and some small shell scripts.

This lists all the articles, starting after a given offset, or zero by default:

      echo -e '\\c 2000 2000\nk)select i,title from .:`:t.kdb where i>'${1-0}'\n' \
      |exec q > 1;
      exec less 1;

Then, to select and read an article or its comments I use the ":!cmd" feature in less(1) to invoke an appropriate shell script (actually, an execline script). Quit, and I am returned to less.

Have seen lots of systems for reading HN made by HN readers, using various databases I guess. This is mine, using kdb+.