Hacker News Reader

Comment by andyferris | Hacker News Reader

So where does the data "live"? I was looking at the syntax, it defines predicates for filters and views and so on, but I don't see the "rows". There is this `file.name` and `file.ext` thing - but where do you set them? What type of file does it point to? CSV? JSON? Something else? The docs seem incomplete.

View on HN

The rows are individual Markdown files and the columns are YAML frontmatter properties in those files.

There are some special properties prefixed by `file.` which are implicit to the file itself, e.g.`file.name` refers to the file name, and `file.ext` is the extension.

The base views are defined as YAML in .base files or can be embedded in code blocks within a Markdown file. You can also export the rendered views to a Markdown table or CSV.

Hmm... so I can't use this to render and filter a table with 10k rows without having 10k markdown files?

If I understand correctly, the intention seems to be "curated list of links" which the user can sort, filter, etc when viewing. I guess that's cool, if you use Obsidian lots and have many notes/links - but when I clicked the article and saw the table I was hoping for a "dataframe" plugin for .md (much like how mermaid works, defined in a codeblock) that references a nearby CSV/JSON/etc file.

I often have a lot of .md files floating around "data" projects and a lightweight tabular renderer (with filtering, sorting, possibly editing) would be absolutely killer. Does such a thing exist already?

Have you ever tried the Dataview plugin?

It allows inline blocks in the `key:: value` format, as well as frontmatter-based data (sort of what Bases are doing) and probably even more.

I think what you're trying to describe is a Jupyter notebook but in a slimmer package. Maybe marimo or quarto? Maybe there are already notebook viewers out there (on GitHub?) that only allow view or edit without code execution, if that suits your needs.

Yes, it relies on a Markdown note file for each row and the “columns” are YAML frontmatter and cached metadata for each file.

I am with you on this, I wish Obsidian would optionally allow you to use YAML or some other structured data directly in the fenced code block or base file.

I really, really want something that kind of takes an Obsidian-like approach to local databases, sort of like Excel/Airtable but with flat, human-editable text files that live on your filesystem with a schema driven property editor. It’s kind of a bummer that this gets so tantalizingly close but doesn’t take it to the logical conclusion. I hope they do it eventually or make it possible with plugins.

Yes exactly. In fact, I'd prefer it by built more like mermaid as a _markdown_ JavaScript plugin thing that supports different data formats (not just YAML frontmatter - bare CSV for example) and have it available outside Obsidian (the github .md renderer, VS Code Markdown Preview, etc).

I guess there is a convergence of ideas going on here because I've actually been tinkering on something like this, a Notion-database-like that just uses flat CSV files (and internally reads it into an in-memory SQLite for filtering, grouping and displaying) then schema files for interpreting the data and displaying it nicely.

Here's a little demo of what I've got working: https://youtu.be/LCR9pAc_xn0.

It's currently still very rough and I'm just using it myself but hoping to open source it at some point.

If you have 10k rows this isn’t for you, but 99% of use cases have less than 100 rows.

For example, book list, movie list, customer list, invoice list, asset register, key register… once you hit a certain point, obsidian probably isn’t the right tool anymore. But no reason to go to the monthly SaaS “right tool” at the POC stage.

Obsidian is the pre-step for a larger database: cheap, fast to customise, easy to backup (git), self supported. It’s probably not going to run a company, but it will suit an individual or small startup.

And 99.99% of discussions about scaling are premature optimisation (cit needed). A lot of people spend more time thinking about scaling then entering their data, which probably means the data is smaller than they think! ha

Reply on news.ycombinator.com