If it is worth keeping, save it in Markdown
p.migdal.pl
p.migdal.pl
I prefer RTF for two main reasons:
* I can't express simple formatting such as "make this text red" in Markdown. No, I don't mean "accentuate this text and leave the decision on how it looks to someone else", I really do mean "make this text red". I do a lot of public speaking, and I want to keep to certain conventions which are easy to read fast.
* Most of the time I am writing text, not reading a version after it goes through a formatter, so I prefer to see it formatted on screen. That's really a limitation on Markdown editors, but it's almost universal so for my point of view, it counts.
And if you want an "archival" format that will stand the test of time it's pretty good (so long as your text encoding is readable, HTML will be).
On a `<div contenteditable=true> element, calling document.execCommand('bold') will make selected text bold in WYSIWYG mode. See https://jsfiddle.net/z0umpb3x/12/ for the concept.
The main idea here is that I don't want to know any syntax like RTF, nor I want to use any tools.
If your editing tool changes or if your switch editors, they will all botch your HTML anywhere you make edits.
RTF is basically "it just works," very much like "Microsoft Word Light."
Can you point me to a page in Firefox that I can use offline to WYSIWYG edit a hypertext document without needing to use the developer console to edit raw HTML?
data:text/html,<html%20contenteditable>
You can use another bookmarklet to turn existing pages or files into editable pages.
Btw, genuinely asking, what's your offline scenario? Long loong time ago, I used TiddlyWiki for a brief time period for my temp offline scenarios. Single local HTML, can edit, but not WYSIWYG.
Keep my data out of corporate hands without needing to run a website myself (eg, a WYSIWYG site) while not having to use some different app.
> The main idea here is that I don't want to know any syntax like RTF
Nor do I. What I want is a file format which is long-term viable, and which I can edit in a WYSIWYG editor. The underlying file format being text-based useful as a recovery mechanism if RTF ever becomes unsupported, but it's not intended that one would edit it manually
> , nor I want to use any tools.
While I do not want to edit a file format manually.
As someone who live in terminal emulators (yes, even home), markdown is more WYSIWYG than most other formats (I still think org-mode is better though)
Well, for reading you can always use pandoc with stdout output and pipe it to a pager. There are also fancier options like https://github.com/Orange-OpenSource/pandoc-terminal-writer (disclaimer: I contributed a small PR a long time ago)
But MD still is closest to plaintext because in most cases MD (or a similar scheme), even when not rendered, can be read easily in its plaintext formatting.
I'd suggest an unbroken chain of oral transmission...
Much more slowly and gracefully than any digital medium we have concocted so far (save for core rope memory, maybe).
> even if preserved can become unreadable (e.g. Linear B) without a surviving linguistic community [...] translations produced with century-scale gaps often lose subtext
This pertains to the message and not the support; also, I'll take missing subtext over missing text any day of the week, thank you very much.
https://arcticworldarchive.org/ would like a word
Furthermore, the vault contents are stored on piqlFilm, which in turn uses Boxing barcodes [1]. So, they literally rely on an analog support.
I restm my case.
And yes, a rendering pass is a drawback.
I've tried 2 different plugins, they are a bit finnicky re formatting (I do a lot of bullet points in my various todo lists and a small thing can break whole block of them) and output looks a bit different, but I still prefer them.
Heck, I often prefer reading markdown itself these days just for myself since the structure is already there and visible, I've learned to see those formatting characters as already sort of rendered formatting so not even looking into rendered pane that much.
By initial release dates, LaTeX (1984) and TeX (1978) predate RTF (1987).
> still text-based
The point about Markdown is that you don't need a complex parser[0] to be able to interpret the files - a simple human can read a Markdown file and get the gist of what is going on. RTF has a whole mess of control strings and codes going on that get in the way of a simple visual understanding.
> I can't express simple formatting such as "make this text red" in Markdown.
You can add raw HTML to Markdown which would accomplish this (at the expense of moving away from the "simple plain text", obvs.) There's a bunch of Markdown parsers with extensions for this kind of thing though (from what I can see, they're all not entirely "simple" either which is a shame.)
Is it as easy as Markdown? No. But it should take about an afternoon for a halfway competent programmer to make an rtf2txt utility from scratch that correctly handles > 90% of the RTF files you're likely to encounter in practice.
Some text and <span style="color:red">this is important</span>HTML gives more control - from plaintext through basic formatting to building pretty much anything. This freedom can be a slippery slope - first comes red text, then custom fonts, then some SVG showing the font, and before you know it, you're building an app.
(That's one thing I love about Hacker News - they kept it simple by not allowing any formatting in posts.)
PDFs are great when you need an exact visual presentation. And yes, I keep plenty of things in PDFs. However, it is not a format I convert to too often—usually, I prefer the flexibility to reflow text for different screen sizes. YMMV.
As for RTF - I'm not sure about its current niche. It's a Microsoft proprietary format with TeX-like syntax, so for custom applications you might need to write your own parser rather than using standard tools like XML parsers. XML feels much cleaner to work with compared to the jungle of other formats we used to live in.
That said, if it works for you, great! I'm just curious why, in this case, you prefer them to a subset of HTML.
A drawback of markup languages, including TeX, is that they intermingle content and style instructions.
> RTF ... is a Microsoft proprietary format
RTF originated with Microsoft, but it's widely supported by scores of word processors and editors. I generally use TextEdit on a Mac, which supports plain text, RTF and RTFD directly. Hence you don't write your own parser: you use the editors already available. It has been around since 1987 with a high degree of stability in the core functionality, so it's a reasonable expectation that those editors will be around long-term, but if they disappear, you can get the text out.
> It's better to give each device (desktop app, ebook reader, printer) flexibility of display
For some purposes, perhaps. Not for mine: if I set text to be red, it should be red on the screen, and red on the paper. Not "emphasised" - red.
PDF is not intended as an editable format, so not relevant here.
Markdown was never really about codifying anything existing. Sure, it took a lot of guidance from existing conventions, but it also invented quite a bit, ignored quite a bit, and compromised quite a bit because of wanting to mostly sit atop HTML.
The most notable deviation from custom is its link syntax, which is simply bad. Where any delimiter was necessary, the longstanding custom was to delimit with angle brackets; and Markdown did allow <https://example.com/>, but its text-with-hyperlink syntax of [text](href) is highly confusing, leads to frequent errors, and makes the huge mistake of using as its delimiters characters that are valid in URLs. (In current spec terms, parentheses are in the set of URL code points <https://url.spec.whatwg.org/#url-code-points>.) This has led to all sorts of trouble. My guess is that angle brackets weren’t used because of the potential for confusion with HTML tags, though there would have been no real parser ambiguity, and autolink syntax kinda messed that argument up anyway.
Then its image syntax of , that’s just plain nonsense.
I’d say that reStructuredText (from a little earlier) matched existing conventions slightly better than Markdown, in general. And it was much more sane, as a language, especially when you wanted to extend it.
> It’s meant to be human readable first and only incidentally convertible to actual markup.
If it were incidental, you wouldn’t use it—just go informal and write what suits you, it will be nicer.
No, the only reason for Markdown is so that you can write HTML with a nicer syntax. It’s not incidental at all. Sure, that syntax is intended for human-readability and -authoring, but it’s essential to Markdown that it is actual markup, poorly-defined though it be as a family of lightweight markup languages <https://en.wikipedia.org/wiki/Lightweight_markup_language>.
> Plain text is the presentation format
Do you know how many README.md files there are on GitHub that are essentially HTML soup? It’s a bit sad, really. It doesn’t invalidate the intention of what you’re saying, but I would say that Markdown is noticeably less about plain text being the presentation format than it was twenty years ago.
It is standard enough but I really, really wish static websites would support it. The fact that it won't creates a huge split in my universe.
"Use RTF instead of Markdown" is one of those.
Depending on what one means by “Markdown” and how completely you want to support RTF, I might argue RTF is way simpler than Markdown. Certainly it’ll be more consistent.
Markdown is far better suited for human authoring; RTF is a data format, not a markup language. And you can cut a lot of corners with Markdown and it mostly won’t bite you. But if you want to do things properly, Markdown gets rather complicated, whereas RTF stays comparatively simple after the initial parsing/serialising cost, especially if you exclude newer features like XML markup.
A lot of people are assuming that you would edit RTF manually. There's no earthly reason to do that.
For file-tagging, I would consider TMSU [0] instead of writing bespoke tools. (ideally we would just use xattrs, but the world isn't ready for that)
[0]: https://tmsu.org/
Making * always be used for bold and _ always for italicizing is so much clearer, and some Markdown flavors (notably WhatsApp) do this. So you only have to do *haha* or _haha_, which also makes italic-bold more _*intuitive*_.
Similarly they should have gone with one style of headings, probably with #.
This frees up more visual clarity. Because you are no longer using *** for bold-italic, you can use that for lines, instead of both --- and ***.
This then further frees --- up to be used for tables.
Although I imagine there's a decent subset of people that uses the alternate style of doing headings === and the 'normal' way of doing lines ---, which would have killed adoption.
And good luck convincing people to adopt a new variant at this point. "Commonermark"? "Peasantmark"? "Rabblemark" actually sounds decent.
Edit: actually, having checked the discourse around it a bit more, Commonmark wasn't created as "one Markdown to rule them all", but rather as "Venn diagram markdown with the most overlap".
> Markdown files are essentially plaintext with some extra syntax for common elements like sections, bullet points, and links. The format deliberately avoids precise control over display details like font selection. Following the rule of least power, I consider this limitation a feature.
One of my biggest ongoing frustrations has been MDX - a sort of markdown-and-JSX mixture whose spec is now in its third release and which has made very little effort to maintain compatibility with either CommonMark or itself. It is fairly strict and fairly elegant, and moving to a new version requires rewriting all previously-written documents to eliminate no-longer-supported syntax and re-training writers. Both of those things are miserable tasks; it has absolutely killed any tolerance I might have had for a stricter parser.
Wonder if any of the LLMs could do that for you?
This is probably to support potential ambiguities and intraword emphasis e.g. underscore is a common pseudo-space so doesn't support intraword use but * does e.g.
is_not_italic
this*is*italic.
It's way harder than I expected because of the combination of the ambiguity of * and ** in multi-symbol runs which support infinite nesting even of the same type of emphasis. A given delimiter run could be many different permutations of plain text `*`, `em` and `strong` depending on context of other delimiter runs that might open and close sections along side other context like punctuation, intraword-ness, flanking and whether sums of runs can be be factored by three!
https://spec.commonmark.org/0.31.2/#emphasis-and-strong-emph...
I never expected "**" could be nested emphasis instead of bold so interpretation requires multiple passes to break down delimiter runs and match them up e.g.
***this* and that* -> *<em><em>this</em> and that </em> is_not_italic
this*is*italic.
this_is_italic
is\_not\_italicI can see it as a choice from the "plain text first" philosophy i.e. the things you typically write in plain text should not need escaping. My intuition pump is that you can copy-paste an email into .md without edits or surprising rendering.
As such, it's doomed to never satisfy everyone. Personally I never use intraword emphasis and I typically only have underscores in non-code names i.e. `this_is_normally_code`.
That leads to mixing presentation logic (meta data, ToC) and content. When typesetting the Markdown, the ToC can be derived from headings and meta data should be isolated to avoid duplication. The following videos demonstrate some of the advantages to this approach:
* https://www.youtube.com/watch?v=cjQ-dle-tAE
* https://www.youtube.com/watch?v=3QpX70O5S30
See my editor's screenshots for more details:
https://keenwrite.com/screenshots.html
My FOSS editor is a cross-platform CLI and GUI application that replaces the shell scripts developed in my blog series about typesetting Markdown.
https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...
Though I wonder what benefits it has over just plain symlinks?
But primarily, I treat them as plaintext files. If I needed to remove frontmatter at some point, it would be a simple script. For any feature specific to a particular Markdown flavor, preprocessing, or system - I expect it to work only as plain text elsewhere.
Also, thanks for sharing about TSMU! I was thinking about similar issues—for example, a photo can simultaneously be "from 2022," "from a conference," and "emotionally important." This doesn't work well with typical nested filesystems, where we need to decide on a single folder hierarchy rather than allowing us to filter based on need (as we can in SQL).
GitHub-flavored Markdown is so popular because you can really easy inline them. You don't have to worry about storing them, linking them correctly, and you can even paste to the Markdown field.
There is no elegant solution like this in actual Markdown.
The formatting is interesting, but it’s not revolutionary or anything.
- Diagrams
- Math
- Any custom blocks - like Figures, algorithms, image boxes, etc
- Numbered chapters / sections (Eg Chapter 1, Appendix A, etc).
- Semantic references
For readmes, thats fine I guess. But I miss all those features when I'm writing proper blog posts, articles and documentation.
There are various hacky workarounds. But as soon as you start using bespoke markdown extensions, you're locked out of 95% of the markdown tooling out there. And everything feels so janky.
I'm looking forward to Typst's HTML output getting more mature. Typst is the only typesetting tool I've ever used that is both enjoyable to use and powerful enough for the kind of documents I want to write. It manages that by being a full on programming language. You can define variables and write custom functions for reused blocks. And there is an ecosystem of 3rd party typst packages. For a paper I wrote recently, my benchmarking tool spat the results into a JSON file. My document loaded that JSON data directly, and used the benchmarking results to populate charts and tables in the paper. It was crazy cool.
but thats added 'on top' of basic markdown, so when you open your markdown file in some other program it looks weird
People just started doing all kinds of extensions and for me that’s silly.
Fine. But in that case, markdown is the wrong tool for blogging and documentation. I want to write rich and interesting content. Markdown is anaemic.
https://github.blog/developer-skills/github/include-diagrams...
Does it work at all? Yeah. Does it work in my markdown editor? Probably not. Does it work in my markdown renderer? I don't know. Which version of Mermaid does it work with? Probably a different version of mermaid on every platform. Can I save my mermaid diagram to a file and link it instead of inlining the mermaid diagram inline? Who knows. Flip a coin everywhere mermaid is supported.
I tried pushing a markdown renderer to the limits once - only to find out that the markdown renderer I was using doesn't correctly implement commonmark, and my markdown file breaks with every other markdown renderer I've tried it with. To say nothing of the custom extensions I tried to use.
At this stage I'd rather keep my markdown files simple, and use something better for real documents. Something like typst.
> GitHub-flavored Markdown is so popular because you can really easy inline them.
I'm not sure what you mean. GitHub-flavored Markdown has pretty much exactly the same image syntax as every other Markdown flavor.
Honestly, I think using an html image tag is the right way to go. I type in markdown all the time, and I have no problem making links. But markdown image syntax I have to double check each time or let the editor figure it out. HTML image tags, I find easier to remember and read than a markdown one. (But maybe that's because I learned HTML before markdown).
I literally have a footlocker filled with old disk drives (remember, since 1979!) and I have never, ever gone back more than a few years, hell, more than a year.
Now that disks are big, I keep a lot of old stuff. I have, eg, screenshots dating back to 2015. Email before then. And so so much more.
I have never gone back more than a few years.
I will continue to archive because I must but, Old Person to Young People... Don't put too much effort into long term availability. It's not a good investment.
Similarly, I'd say save everything, but spend no time on organizing it, relying on search and ai/future technology to find what you want from among the mess.
I had a private mailing list for 15 years and had emails squirreled away across several hard drives. I archived them all to my Mac, under a directory under /, and tossed the disks. Was too broke to have another disk for backups.
Then Apple decided in a upgrade to trash everything not-Apple under /. Archives gone. No warning. Really amateur move by them. Grrr.
For me, everything swirls in an enjoyable vortex towards org-mode.
- Literate Programming, tangle/weave
- Export to DocX, PDF, HTML
- Org-Roam
- Time Management.
Several things mentioned above are day to day. I think spectacular things are often made up of collections of useful everyday things.
It's just a shame that org format works really well only in emacs.
They’re fine.
IIRC the trick was to get a pipeline for Markdown to HTML, render it into a WYSIWYG editor, then convert the HTML to an AST, and walk that to generate the markdown. I had to “format” both the markdown and html on each render (bidirectional round trip render) because parsing/gen wasn’t whitespace stable.
https://system3.md/observatory/categories/collaboration-team...
Peerdraft, Relay, and Screen Garden are all based on CRDTs, and Obsidian is also currently working on native collaboration.
(disclaimer: I work on Relay)
We still have a centralized service for authentication and authorization, but if you self host it is impossible for us to access your files.
Surprised , nobody mentioned qownnotes
Edit: I was wondering how to enable this mode because it wasn't in my qownnotes ,Here's how I found it , go to the help section , click find action , and search preview and click on show note preview panel.
Now the caveat is that if you want to see it blitted , you have to save the file once to see it automatically show in the other side. Maybe this can also be definitely automated / I feel like there was some feature that did that for that as well or atleast its very non trivial.
Edit 2 : okay so I just realized that qownnotes also ships with autosave feature which saves and thus also shows what you type in reader mode in like a 0.5 second delay. And I think there is also a way to decrease / increase the autosave part as well
Dude , I didn't realize it , but qownnotes is so good!
- [HackMD: Your Collaborative Markdown Workspace for Knowledge Sharing](https://hackmd.io/)
The only markup I'm finding with a grammar is MediaWiki (sort of..)
https://www.mediawiki.org/wiki/Markup_spec
Even Djot doesn't seem to have one. Weird..
And yet almost no one has...
The problem is the lack of READER applications to simply view (not edit or convert) all these Markdown documents.
But... come on. You might just as well say any text editor is a browser, because you can technically read HTML with it.
You can also technically read Word documents with a text reader.
A simple grammar probably really limits how you can design your syntax.
Here is an example of the problems:
https://www.mediawiki.org/wiki/Alternative_parsers
There is only parsoid, developed for the visual editor and that took pretty much a decade to build with much pain and suffering.
This is not the answer.
What do you recommend?
It does actually seem that djot has a grammar of sorts..
https://github.com/treeman/tree-sitter-djot/blob/master/gram...
(it's designed for a tree sitter.. I'm not super clear if it's globally usable)
The long-term solution is having whatever markdown grammar you want and converting it to a standard AST. Then anyone can create their own transformations of that AST to render whatever document they want, including a new markdown document potentially in a different grammar.
https://pandoc.org/using-the-pandoc-api.html#pandocs-archite...
So while my notes may rely on some personal scripts to get there most value out of them, I strongly value that they are still plain text and I can always move them into a new workflow if I need to.
If anyone has a good solution I'm all ears.
Commonmark doesn't even mention "bold", "italic", and "underline". It just says "emphasis" and "strong emphasis". You can style it however you want.
So the purpose of , * etc is purely emphasis. If you need to represent something specific (bold, italic etc) then that’s a job for the Markdown parser (or embedded HTML etc). The result of the parser (HTML, etc) will be less human readable, but actually able to specify formatting.
I agree that CommonMark could be extended, but I think the focus should be on semantic* relevance rather than markup specification.
Could you use the convention in your documents that "_" is the underline delimiter? I know that the default is to render it as italic/emphasis but that is just a decision at rendering time. The semantics of emphasize/underline could easily overlap.
Of course if you want 3 levels of emphasis with bold, italic, and underline, then yes you need to look elsewhere.
Markdown isn't really a formatting tool. it is a way to structure text in the minimal way that a person would interpret it and a machine could render it.
Relying on format-specific tags imposes stronger constraints on endpoints and/or increases complexity of your document build process.
If you want those outputs to faithfully represent formatting, you either need to juggle multiple inline directives for each desired output format, or find some universal Markdown-based mechanism for achieving the same result.
I'd like to make clear that I'm familiar with Markdown; the fact that its original design intent was streamlining HTML generation; that inline "native" code is a feature, not a but, but all the same a rather fraught one; and that actual practice has moved far beyond Markdown merely being used to generate HTML, least of all my own such practice.
I've discussed this situation previously on HN (ironically from the PoV of using LaTeX embeds within Markdown creating problems when attempting to generate other-than-LaTeX outputs), see: <https://news.ycombinator.com/item?id=29690056> (2021).
And I'd asked about the HTML and/or LaTeX conditional generation in a StackOverflow post about seven years ago: <https://stackoverflow.com/questions/4820502a9/pandoc-have-ei...>.
I don't know if this helps you, but you said "older": in the 20th century world of typewriters--which had no italics--underlining was used as a substitute for italics. Transforming underlines to italics or going the other way was considered normal. You wouldn't use both in the same document.
Both matter, and although Markdown flavours handle the notional case well, they fall down at this (and several other) typographical capabilities. Expressing text in a particular colour (or greyshade) is another example. It's possible to achieve this in practice through embedded HTML and/or CSS tags, or through augmented Markdown variants (Pandoc's Markdown can achieve some things CommonMark or DaringFireball Markdown cannot).
Ultimately though I find I need to switch to a more capable and consistent text-layout engine, usually LaTeX in my case.
Though for even quite large and modestly complex works, Markdown is either sufficient entirely or is useful in getting the work off the ground before switching to a more powerful option.
converting old typewritten notes, they may contain typewriter underlining, and it may represent italics. Markdown would be entirely sufficient to handle that.
there was no need to de-clarify my comment.
Unlike handwriting, typewriting uniform (both in type and spacing), and markedly faster.
Unlike printing, typewriting is limited (generally a single typeface, no variability in face, size, or styling (e.g., roman, bold, italic), and requires further guidance to define specifically what result is desired where a typewritten work is not a document's final form.
It's worth noting that print itself differs from handwriting: when we write letters, forms and sizes vary, different writers often differ markedly in their own scripts, trained copyists may achieve a high level of standardisation, but that itself requires significant training and is achievable only by a limited number of artisans,[1] and letterforms themselves are not discrete but individually instanced each time they are created. With the advent of moveable-type printing,[2] letterforms became fixed, and with digital typesetting and computer fonts, each discrete shape or language-specific forms, say, the Roman A, Greek Α (alpha), and Cyrillic А (Azǔ/Азъ), are represented by distinct code points, but are nearly or entirely indistinguishable when rendered on-screen or in print. Further, over the history of both handwriting and typesetting, conventions have emerged for the textual representation of language, including spacing of words (versus scripto continuo), punctuation, paragraphs, page numbering, division of books into chapters, sections, parts, subsections, etc., of lists, tables, indices, (foot|end|side)notes, (parenthesis), drop-caps, figure captions, cataloguing, etc., etc. All of those were inventions and conventions not inherent to language, writing, printing, document preparation, or archival and retrieval themselves. There's still considerable variation between different print language representations, e.g., many texts lack equivalents of italic, bold, or even upper/lower case letterform distinctions.
Typewriting itself occupies an interesting space, being a primary endpoint for some types of documents (correspondence, forms, and the like) and an intermediate form for others, most notably published articles and books. Given that typewriting has both capabilities and limitations which aren't present in typeset documents (whether moveable type or digital), it's not possible to draw a distinct correspondence between what a typewriter outputs and how that might be represented in a derived document. Yes, typewriters can generate underlines, but that might be represented in typeset print as italic, bold, underline, or something else entirely. In practice, editors proofing marks were inserted (as handwritten notations) on a typed manuscript to indicate the preferred presentation, generally following the author's intent and/or the publisher's own house style conventions. See: <https://en.wikipedia.org/wiki/List_of_proofreader%27s_marks>.
________________________________
Notes:
1. An anecdote which sticks with me: among the 1001 Arabian Nights stories is one in which a character makes specific references to the not only his literacy and scribal capabilities, but the types of scripts he could produce. That is, this was a specific and valued skill of that age worth noting, even in a general-audience work.
2. As distinguished from earlier monoblock printing in which a whole work was engraved on a wood block or metal plate, typified by early Pamphilus, seu de Amore from which we have the word pamphlet, see: <https://www.etymonline.com/word/pamphlet>. Such monoblock prints were more like a photocopied handwritten letter, in which variations in individual letterforms are replicated, than they are standardised print obtained from moveable type or, more recently and familiarly, computer-based digital typesetting or Web documents, in which fonts are standardised and each given character is identical to all others matching that style.
I use FSNotes today on macOS and iOS. Both apps are open source, both use well-structured .textbundle directories that separate Markdown content from JSON metadata and binary attachments. Synchronization happens through Git. It's a very powerful combination.
Ironically, I wrote a blog post some 8 years ago about this very subject. That blog post is now offline.
My biggest beef with org mode and all of the markdown apps I've tried is the asset management problem. For me screenshots are almost as important as the text part of the note, and are usually strongly tied to a single note. I've taken to using apple notes at work just because it "solves" that well enough, but I'd really prefer to work in markdown/plain text (except for the images).
Here's a demo https://www.youtube.com/watch?v=SykbiVweYH8
Ultimately I'm trying to index my "window" to the web as embedded content in a vector store. Not sure exactly what I'm going to do with it yet but I imagine it will be a component of some kind of personal agent system I can use to reference old info and help as a writing tool or as an "idea generator" of some kind. I'll likely end up not using most of it but you never know.
I've scraped about 10k markdown files which has created a ~10gb chromadb instance so far. Eventually I'll probably create separate collections based on domain, and filter down items that I care about more.
That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus.
The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of.
I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use.
I develop a tool called HamsterBase that seems to address a lot of these issues we've been discussing. t's a local-first app. That means all your data stays on your own device - no need to worry about your personal archives being stored on someone else's servers. There's no sign-up or registration required, which is refreshing in today's cloud-centric world.
I don't share that experience. I typeset all these documents using Markdown with pandoc's div extension, transformed into XHTML, and then passed to ConTeXt:
* https://impacts.to/downloads/lowres/impacts.pdf
* https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdow...
* https://pdfhost.io/v/4FeAGGasj_SepiSolar_Highlevel_Software_...
From XHTML, the document is transformed into TeX statements, which opens a world of possibilities. In the following video, custom styling is applied to nested contents:
Alternatives for authoring PDFs include LaTeX or similar markup languages, or GUI-based tools.
For many works, Markdown is more than sufficient for producing book-length texts (I've done this numerous times myself, either authoring my own works or transcribing/modifying books for improved access/readability). Markdown's benefit is that it is extraordinarily lightweight, and removes overhead from the authoring process.
Even where one ultimately chooses to migrate from Markdown to some more capable authoring format, Markdown remains useful for creating the original rough form of the work. Complex elements (figures, formulae, tables, etc.) can be indicated and, after document conversion from, say Markdown to LaTeX, fleshed out in full.
With tools such as Pandoc (see my earlier comments on it), it's trivially possible to create multiple outputs (I usually refer to these as "endpoints") of a document. I've used Makefiles to drive this process, such that I write source in Markdown and generate partial or full HTML documents,[1] other LWMLs,[2] PDF, ePub, straight ASCII/UTF-8/Unicode text, word-processing formats, etc., as I want. The set of Markdown + Pandoc makes this trivial in ways that, say, LaTeX alone isn't entirely suited.[3]
It's of course possible to use another LWML as the source format. Markdown has its limitations, but is most widely known and implemented, and limitations workarounds are typically reasonable.
________________________________
Notes:
1. A partial HTML doc may be useful for dropping into a larger document, and doesn't require global HTML elements such as the <html>, <head>, <body> tags, or others such as <nav> or <aside> in most cases.
2. Lightweight markup languages such as bbCode, AsciiDoc, RST, MediaWiki, OrgMode, etc., etc., see: <https://en.wikipedia.org/wiki/Lightweight_markup_language>. Useful when inserting the document into systems based on these formats.
The publication of all those as PDFs INSTEAD OF Markdown testifies to Markdown's big problem: the lack of readers (viewers) for it.
"Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions."
Any text editor (Notepad, TextPad, (neo)vi(m), Emacs, TextMate, Apostrophe, GhostWriter, Typora, etc.) will do. Markdown-specific editors have either a real-time preview or the ability to edit as WYSIWYM:
* https://keenwrite.com/ (mine, FOSS, cross-platform)
* https://pandao.github.io/editor.md/en.html
* https://markdownlivepreview.com/
What do you mean by lack of readers?
Google Docs now supports Markdown files, so if I need to convert the Markdown file to Word or PDF, I just open it in Docs and download it in the format I need. (Pandoc also works for this, as the author mentions). Converting HTML to Markdown can also be done in Docs: copy and paste the web page text into Google Docs, and download the file as Markdown.
For mobile, I use the DriveSync app to download my notes (Markdown) folder to my phone. Then I use Obsidian to open and edit the files.
Very easy to search notes and even have a dedicated folder for diary entries.
Some of my scribblings are useful or important enough to get added to the markdown files and printed the next time.
- https://markdownreader.github.io - https://github.com/simov/markdown-viewer
Those two are quite good.
After years of looking, I finally ended up with Marked (for Mac). When you ask for a Markdown reader in any forum, you get nothing but suggestions for EDITORS, which happen to have a preview pane. But what is it "previewing," when everybody's just reading these things as plain text with the formatting codes embedded in them?
Sure, keep your archive text in markdown (which one? a dumb person asks). But I'd recommend managing it with org-mode, it doesn't really care what format your text is in.
(Yeah I saw the footnote mentioning org-mode but that reads to me that org-mode's reference there is entirely about the markup flavor.)
Looking back I can't believe I considered just bookmarking a link enough to save it long-term. Sure, I lost a lot of cruft but there were some gems that in retrospect I'd have liked to still reference or look at today. Eh, hindsight is 20/20 as the saying goes.
I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
Idk, I think if it's worth saving it's worth saving and the only person who can determine if it's "worth it" is me.
I agree that some people have an obsession where they save data that isn't worth it, but r/DataHoarder is a great place with a lot of information on building and maintaining large data systems for hobbyists, regardless of what you actually store.
Also, some people other than yourself might turn out to be a better predictor of “future you” than “current you”.
So many people are hyper individualistic and forget just how predictable individuals can be to others!
The thing is, people say this, and I am sure for some amount of content it's true. However I eventually realised I have never had a single issue if required, in retrieving literally any piece of software or digital content after the fact from somewhere on the internet.
It's pretty much why I care so little about what happens to my steam library when Gaben kicks it. If I get the urge to replay something in twenty years that I paid $3 dollars for and its suddenly gone, i'll just go find it elsewhere.
Once I had gotten pretty much everything I wanted from running a large scale storage system (largely to learn the in's and outs of linux/general storage concepts) I pretty much just gave it up. Its a lot of money to hold onto things that at this point, I pretty much know i'll always be able to recover elsewhere. I'd rather someone else pay the electricity/drive cost for me.
Sure. And I appreciate it. Like I appreciate the internet archive etc.
I don't know that the economics of having multiple Internet Archive-like organizations is currently feasible (I imagine getting funding for one of them is hard enough), but even a partial offline mirror hosted someplace else would be nice. Maybe to save space they could take the oldest version of a page, the newest, and the midmost version timewise, discarding all other versions. They could also heavily compress images, video and audio to save storage space (would increase processing costs, but if willing to throw out quality, could compress quickly and still save a bunch of space. E.g. downscale all videos to 480p and use veryfast preset and CRF 28 with ffmpeg. Even 240p is a lot better than nothing. A pixelated form of history is better than no history.)
Information is removed or altered constantly and usually, I cannot find anything on the Internet Archive either. For whatever reason WayBackMachine, for my use cases, is nearly always blank.
But I look for semi-obscure publications and statements from (nation)states and organizations.
For me, everything swirls in a lovely vortex towards org-mode. - Literate Programming, tangel/weave - Export to DocX, PDF, HTML - Org-Roam - Time Management.
(Don't answer that - there's always one asshole who would)
MW gets you a massively scalable doc store that does not need much room. Most MW instances are MySQL/MariaDB backed and the schema etc is very well described.
Keep it plain text for "notes" but a MW will be easily discoverable for quite some time from now.
encouraging..
Can we have a damn math keyboard and proper character encoding instead of doing shenanigans with latex / office equation editor ?
Why in this exact text box I cannot type a differential equation ?
Tables, in particular, just suck, especially if you want to have even slight formatting inside of the cells.
Unfortunately, it’s either plain-text-readable or rich representation. Pick your poison.
There are very, very few. I use Marked 2, for Mac. I don't even remember if I ever found another one. It's irritating as hell, because pretty much every open-source project's read-me files are in Markdown. Why, when there is no viewer anywhere near as ubiquitous as those for PDF... despite Markdown being much simpler and better understood?
I have tried MarkText, which is yet another editor with a viewer but it's free.
<https://pwmt.org/projects/zathura/>
This lets you work on the doc in a terminal window and have the (reasonably constantly updated) formatted output in a PDF viewer.
Short documents will render virtually instantly. I've not had long renders until documents extend to at least several chapters worth of text if not book-length, and even then it's a matter of a few seconds in most cases. Highly-formatted texts may of course take longer.
I happen to generally be aiming for PDF output, it's among the more complex to produce, and still runs pretty much without concern even on very large documents. Anything else would be even more trivial.
I have one of those big dvd "catalogs" that takes 4 discs per side of a page.
Keep one at home and one at my parents' place.
I trust them more than usb-sticks. Though that may be irrational.
But the time for burning files to dvd seems almost over. It is hard /impossible to buy a computer with a dvd drive.
That is no problem for me since I have a collection fo externals as well as internals. and life is good now since blank dvd media is cheap .
But again, you need a dvd reader, and in the future, that may become difficult.
In my experience hard drives, USB sticks fail and regular hard drives fail.
It has been many years since I have had any involvement but backup tapes probably have issues as well, but the rapid production of new tapes and new formats is an issue already
I dont have any data to evaluate the best choice is SSD drives?
No matter what technology is picked, at some point to preserve the data it needs to be migrated to whatever comes down the line.
What I have gotten to work was to download a file from the sync app, open it in a markdown editor app and then save it to the cloud by sending it back to the sync app. It technically works but it was a bit too inconvenient to become a real habit (too many taps, need to rename the file on upload and set location each time,...).
With the AI coding tools getting better each day, I'm starting to think why I would spend any time researching what's out there for what I want, instead of just using an AI coding agent to put something together in 10 mins, and forget about it.
It's getting easier and faster to have AI build something that solves my exact problem. Maybe not perfect, but OK.
I save everything interesting. I have a data folder with letters a-z in it. Something interesting might be saved in HTML or PDF under data/a/ai/programming
Folders have a problem because the same thing could be saved under data/p/programming/ai
But it is a start. For everything else, there is recoll. https://www.recoll.org/
I used to use Joplin, lately switched to Obsidian. Both offer this functionality.
I'm tired of basic features being missing and extensions breaking because they're no longer maintained, and basic features like linking between notes while writing a note not being built in.
During my long sabbatical, I wanted to take notes on my phone, a LOT. Joplin sucks at that, clumsy, non-user friendly android client.
Tried obsidian (first on mobile) and it is superb. I had to install a couple extensons (S3 sync, "Ink" for drawing with a pen), and it just works. It's so good, I sometimes even edit tables on my phone. With Joplin, note taking on my phone was just dumping thoughts in random formats to it and later fixing it on my desktop.
Like Joplin, Obsidian also has a Firefox extension to capture a web page I to markdown.
So after a couple days of trial, I realized that all the features Joplin has, obsidian has it too, with a much better (and snappier) UX on both my Linux desktop and Android. The only thing I wish for of it was Open Source. But oh well, I'm not dogmatic about that anymore
Imagine writing everything in HTML, but there are no browsers to render it. That's basically where Markdown has been forever.
I know what they mean - "running applications that you maintain and deploy yourself, on hardware/platforms that you don't" - but this is strange, to my eyes. If it's running on someone else's hardware (whatever it is), then it's not self-*hosted*, surely? It's self-owned, but not self-hosted?
That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus.
The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of.
I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use.
It's an interesting problem space, and I think there's a lot of room for innovation in how we approach personal web archiving and knowledge management.
I use VSCode for markdown.
Obsidian's been coming up on the radar often.
This post finally made me try it out.
I like it a lot.
But there's one reason I won't be using it as my main driver for markdown files: I can't open files that are not in a vault. I have markdown files everywhere on my drive. And I don't want to make the entire drive a vault (for various reasons).
Obsidian configurable as...
1) my default file handler for markdown files
2) capable of opening and saving markdown files in any location on my PC
...would be sweet. (From my research, it can't do these currently.)
Just pick a more relevant format for contrast to see that this is no feature! It's not like PDF is the only alternative
I can get by just fine with that system, but a handful of months back I started wanting software again. Reminders, task wrangling, workflows around taking meeting notes, taking and processing transcripts of talking through ideas, automated daily and weekly checkins with summaries, project work logs, managing lists of things to talk about with people, the list goes on....
Same reasons I have always reached for software, and the same reasons I wrote my own system a few times over. But this time I had some new thoughts:
- I want this to have a chance at being my last system. For that, I must be able to read/edit the data without special software. I settled on committing to building software that interfaces with folders of Markdown files exclusively. I could use Obsidian to cover any gaps and get work done immediately–I don't need my software to do it all right away.
- I want to own as much of my recorded activity/thoughts as possible, so I can drop it into new AI models, giving them a ton of context about me and what I'm up to, and avoid getting vendor locked to OpenAI.
- I want ubiquitous access to the system, which means it's gotta be easily used from a phone.
7k LOC later and I've got a Telegram bot with a plugin architecture and a pile of plugins that implement everything I've described and more. The plugin arch means there's a defined interface and every new piece of functionality never ends up with more than 1k LOC in a file. My objective was to structure the project specifically so I could avoid the pitfalls of AI generated code as projects get large. Everything isolated with well defined integration points.
I chose Telegram because they have a great API, supporting custom keyboards for quick actions, audio input for taking voice memos that my system transcribes, and reaching out to me with reminders/requests on whatever device I'm on.
The result is thousands of messages that have translated into a nicely organized Obsidian vault. Couldn't be happier and think there's a chance I'll live with this thing for the foreseeable future–and I can always swap out the interface away from Telegram, build a proper frontend, or drop it altogether and be left with my Markdown files.
If anyone is interested I'd be happy to share what I've got. Just my private project that I'm reaping a lot of benefit from.
Here's a quick dump of some of my plugin commands to get a flavor of what I'm talking about: https://gist.github.com/zackham/3c2d061e6dd0127958c913329aa0...
text.txt
Readable in everything, since forever.