wasimlorgat

Very cool how you pieced together these OS libraries and models! TIL about PythonAnywhere. Would it be possible to have your task/script update the index.html file? Since that’s the only thing that updates the feed, it might be a nice little simplification to render it without JS.

Thanks for sharing! Hmm, it isn’t clear to me from your post why a library would be harder to maintain. Could you share more? The database seems to add quite a bit of complexity at a system-level and I’m still not sure the benefit.

Qdrant looks great, thanks for sharing! I'll definitely play around with it and do some benchmarks :) Have you used it / compared it with faiss yourself?

I'm always surprised when people advocate for .py files over notebooks because of poor software practice. (Genuine question) have you found that it improves the situation at all?

Oh, on the topic of file formats: Quarto also lets you do plaintext notebooks in quite an interesting way, definitely worth checking out: https://quarto.org/docs/computations/python.html

Have you worked with Jupyter notebooks and git? It's a literally true statement :D and quite a struggle for many of us

It does an ordinary three-way git merge (treating notebooks as plaintext) then a two-way merge on conflicted bits. We opted for that approach because its incredibly simple and has worked perfectly for us (I think since we tend to work with small code cells). I think nbdime has a full-on three-way notebook merge if that's what you need, which can be used together with nbdev's Jupyter save hook to clan up unneeded metadata.

I agree re format vs tool complexity. I don't think Jupyter is a particularly difficult format though, its mostly light JSON -- all human-readable.

We realised after working with Jupyter+Git for a while that the pain-points were actually with Jupyter editors (and/or their conventions) rather than the format, because they do things like store user-metadata in the file which pollutes diffs and leads to merge conflicts.

In fact, if Jupyter editors could handle merge conflicted files, we wouldn't need a custom merge driver either.

Jupytext does a lot more than just fix Jupyter/git integration, which is great if you want to adopt its approach, but a bit too heavy IMO if you don't. The approach mentioned here is extremely lightweight and doesn't use too much more than built-in Jupyter/git functionality (and it all happens automatically behind the scenes)

Hi, I'm the author of the git merge driver and Jupyter save hook in nbdev2 :) I'd be happy to answer any questions you have about how we're handling using notebooks with git