GNU Recutils – Tools and libraries to access plain-text databases | Hacker News Reader

GNU Recutils – Tools and libraries to access plain-text databases

gnu.org

89 pointsnetworked14 comments

... is ... is the logo two turtles mating?

I don't feel so bad about our GNU Octave logo now.

Yeap: http://www.gnu.org/software/recutils/faq.html#whyturtles

Two male turtles actually.

Oh, very nice. I've been using org-mode lists with ad-hoc "key: value" syntax inside of them for some kinds of stuff. This looks better structured, while still being friendlier to write than something like XML or YAML.

a related tool for keeping track of what you've been up to is Brett Terpstra's Doing: http://brettterpstra.com/projects/doing/

The only think I feel is missing is traditional SQL syntax. This makes it much harder to use as a drop-in replacement where someone would actually otherwise use the competitors mentioned on their homepage.

I think its real competition is SQLLite and JSON / YAML, and it definitely has some benefits over the text formats, it can't compare with SQLLite.

There are applications that let you query structured text with SQL. For example, https://github.com/dinedal/textql there are others as well.

There is a lot of tools of this kind [1] but nearly all of them work with CSV or other delimiter-separated data. What made recutils stand out for me is that it has tools to modify files in place, not just select data from them, and its file format is different.

The rec format is interesting because it is a specification for what others have used before without one. It seems quite readable and does not suffer from the ",,,,,," problem of CSV.

[1] Here's the list of ones I have run into so far:

* https://github.com/harelba/q

* https://github.com/tobimensch/termsql

* https://github.com/samuel/squawk

* https://github.com/tjunier/sqawk

I've also written one: https://github.com/dbohdan/sqawk. It turned out I wasn't the only one to come up with the name.

"The only think I feel is missing is traditional SQL syntax."

Presumes everything is relational? I notice the mention of 'advanced database' technology joins, and then I notice this example:

  $ recsel -e "Dob >> '31 July 1994' && Dob << '01 August 1998'" -p Name,Email acquaintances.rec

So yeah, I sort of agree, what is the standard unified search language? Is it as easy to learn, use as SQL?

"can't compare with SQLLite."

I'd agree with you on JSON/YAML but SQLLite does one thing each of these format/tools do not, converts "readable" text to binary. Text is good for smaller, less structured data. There maybe good reasons to not want it locked up.

There is one area I didn't see in the docs that would make SQLLite to be really useful. A two-way bridge allowing an export/import of the recutils formats allowing both search via SQL, yet maintaining the text format data.

Thank you for sharing! I just gave it a spin and wrote up a short tutorial here:

http://binarylog.net/code/password-manager-recutils-tutorial...

I foresee many more usecases for recutils in my day-to-day life.

Doesn't mention utf-8, so I'll assume it dies as soon as I type something strange like an ö in my foolish attempt to categorize a thing outside of the US.

Why would you think that? If it is really old that may be possible, but everything halfway current and maintained should support UTF-8 by default by now, especially in the linux world.

And in fact, it is working fine. I took the example from https://www.gnu.org/software/recutils/manual/recutils.html#A... and exchanged Emacs with Ämacs, and the selector works

    ~$ recsel -e "Title ~ '.*Ämacs.*'" -P Title books.rec
    GNU Ämacs Manual

This is a bit worrisome, but they're pretty explicit where they require ASCII, so I'm willing to give them the benefit of the doubt until proven otherwise.

Interesting format, though, with some definite benefits over other text storage formats.

According to the manual [1] the format requires that field names match [a-zA-Z%][a-zA-Z0-9_]* but it allows any characters in the value other than an unescaped newline. For example, the command

  echo "\"country\",population\nÖsterreich,8.58" | csv2rec --strict | rec2csv

works as expected. Somewhat worryingly, the format specification does not say anything about \0. I'd rather see use of \0 explicitly prohibited.

I have found one place where the field name rule may not be enforced properly. It appears that csv2rec will use a noncompliant column name from a CSV file header as a key even in strict mode; however, rec2csv or relset will not accept the output it generates as input. I.e.,

  echo "Österreich_population\n8.58" | csv2rec --strict | rec2csv

blows up on rec2csv rather than on csv2rec (which is what I think should happen).

[1] http://www.gnu.org/software/recutils/manual/recutils.html#Th...

Reply on news.ycombinator.com