Data Engineering Vault: A 1000 Node Second Brain for DE Knowledge | Hacker News Reader

Data Engineering Vault: A 1000 Node Second Brain for DE Knowledge

ssp.sh

123 pointsarticsputnik19 comments

More than a mere collection of terms, this is a curated network of data engineering knowledge to facilitate exploration and discovery. Like a digital garden, it features 1000+ interconnected concepts as gateways to deeper insights.

Key features:

- Covers 100+ core data engineering topics

- Inspired by Second Brain and Digital Garden methodologies

- Includes links to books, podcasts, and thought leaders in the field

- Updated regularly

- Ideal for both beginners and experienced professionals

Happy to get feedback :)

I wish it would be in the form of a wiki or repo that anybody can update. Otherwise thanks for sharing that.

I actually did the other thing too, that you can update and PR here: https://github.com/airbytehq/glossary/. The same approach, but not many did, it. And for me to keep going is much easier if it's just me, as I can spend time on producing valuable content and not merging, although the value of other opinions would be greatly beneficial too.

How do you plan on keeping this up to date?

It's all in my Obsidian Vault. And I update every day. I do it mainly for myself. I have been a writer for 8 years, but I have been taking notes already since 2003 (OneNote). But since I use Obsidian, it takes me zero effort to publish some notes publicly; that's why I'll have an easy time continuing. And I focus on small bites every day, rather than huge updates and try hard. This way it's just natural to me. Whenever I research something or learn something, I add it to my Vault, it only takes me 10 more seconds to put it in the work of research I did already. If I want, I spend some more minutes to add some thoughts and connect with other related terms. But that's where I learn most. So it's really rewarding to me.

btw: I made a YT video of how I update it, in case of interest: https://youtu.be/myHKHM2mIis?si=8bZtmO0cdpItDZV7

Before I wrote about it too: https://www.ssp.sh/blog/obsidian-note-taking-workflow/

second brain has to be up there in dumb techbro things

Having guzzled that Kool-aid myself for a while, and I'm still shaking it off: PKM/zettelkasten/digital garden folks feel a lot like the GTD fad visited upon a new generation of victims.

It has a lot of audience and topic overlap with grindset/hustle culture. There's a large, insidious YouTube corpus promising productivity panacea if only you'll adopt whatever specific software and methodology is currently being blessed. The fetishization of a specific process and artifact is treated as whats important rather than cultivating writing habits and periods of reflection.

Neurodivergent and otherwise chronically-disorganized folks aren't going to find salvation there, but often find themselves suckered in to something like this.

This, sadly, resonates with me. I've been pulled into GTD and a few other similar kool-aid stations, and it's not sustainable and mostly not valuable. It looks cool and feels cool but fails in the value department for me.

I agree that there’s a lot of fetishisation going on. Which is fine, if that’s what you’re into. Graph linking and visualisation in particular seems like a total furfy.

On the other hand, I have derived great value from adopting a practice of writing discrete things I need to remember or refer to later into a folder of date-stamped .txt or .md files and then later surfacing them with a generic full-text search tool. I can’t imagine sitting down to do my taxes, for example, without that information at my fingertips. Others may have more luck retaining in-memory the multitude of small details life throws at you.

Yeah. As someone with ADD who over-medicated with Adderall for a period, this is the kind of shit I'd love to do. Organization for organization's sake. Making lists of lists and links to docs with more lists and connections to spreadsheets, etc. Spinning my wheels I'd call it. Reinventing the internet locally. I wonder how many of these people are eating 30+ mg of Adderall a day. It feels fun, but you're not actually doing anything.

This is sooo true it hurts.

Until you're at a reception dinner, seated beside someone you admire and you're instantly pulling up relevant research from your knowledgebase on topics they care about and are working on.

And then you're doing things, because they're flying you to Berlin and contracting you to do them.

I don't live in a movie though.

how do you think research collaborations start? hint: it's not between complete strangers. so they had to have found the other useful in their own field previously.

> they're flying you to Berlin

What kind of punishment is this supposed to be? I think there's something in the Geneva convention against something like this.

I lol'd :)

well people may need some time to relearn that forgetting is part of brain function for a reason

Generalizing & disparaging technology you dislike or don’t use is weak sauce.

Second Brain is a form of note taking used in paper notebooks, by engineers of all industries for decade upon decades.

An advantage is digital versions can do tagging, hyperlinking, and be embedded with AI & LLMs.

Leaders and inventors like Vannevar Bush and Douglas Englebart pushed towards this.

You may not like it, but others do.

ppl love dumb things doesn't mean those things aren't dumb. Lots of ppl liked NFT monkeys.

Reply on news.ycombinator.com