Link Archive – 03.2023 Update
github.com
github.com
Do you track yourself? Do you check, what you are reading, and verify your own progress? Some time ago I started thinking about my problems with Google search, and was inspired by other project to write a program for myself.
Throughout my life I have found many resources worthy of my time, but which I have lost, or forgotten about. I wanted to change that. I wanted to be able to look back.
I have started my project in Sept of 2023. The core functionality is: to store, browse, and search for links. Additionally I added RSS sources as sources, so every hour I ping sources to check if there is anything new, and I expand my database. Does that replace Google search? Nope. On the other hand it expanded my horizons.
I found most of RSS aggregators lacking:
- some provides ability to bookmark entries, but some do not allow to insert links manually
- some do not provide ability to tag links
- some are on-line, and I strongly prefer local NAS web apps
- most do not provide clean ability to search link database (as it may involve heavy operations)
I found Google search lacking:
- 10 pages each with 10 links for a searched keywords is not enough. If I write 'covid' into google search I would be expecting millions of links in results, from variety of sources. Even Fox news, even daily mail
- I have results curated by big tech, I prefer to curate output myself
What I have found about archive.org:
- it is slow and unreliable
- does not store RSS sources often enough
- author of RSS program objected idea to make requests to archive.org, as there could be many instances of RSS programs, which could be a problem for archive.org
In the process I have realized that I have written something similar to reddit, or digg. That is only resemblance on the surface. The link in this topic is the data. I am interested in links. I would really like to be able to import data from other uses, to see what is interesting and important for them. I like the idea of fediverses, and distributed mechanisms. I think that it could also work for search results.
Can everybody benefit from the links I have stored? I do not know. I know for certain that I am interested only in some topics, and some other people might looks at the world from a different perspective, therefore my links 'could offend them'.
Every day bookmarks and yearly repos are updated.
Projects:
- https://github.com/rumca-js/RSS-Link-Database - bookmarked entries
- https://github.com/rumca-js/RSS-Link-Database-2023 - all captured links in 2023
- https://github.com/rumca-js/RSS-Link-Database-2022 - all captured links in 2022
- https://github.com/rumca-js/Django-rss-feed - my humble app that I am using, which captures the links. I think that the idea could be implemented in any env (php, as plugin to nextcloud etc.)
My database is getting bigger and bigger, and I found that RSS is not a best place for my data, as written in https://www.jefftk.com/p/reverse-rss-feed. Therefore commonly everyday, has its own directory, with data in it. I am using JSON for storage, or markdown for github preview.
Inspirations:
https://www.tdpain.net/blog/a-year-of-reading
https://zanshin.net/2022/09/11/automating-a-reading-list/
https://dkb.io/post/google-search-is-dying
https://www.youtube.com/watch?v=N8P6MTOQlyk
https://www.youtube.com/watch?v=nI3GVw2JSEI
https://www.youtube.com/watch?v=NT7_SxJ3oSI
Currently I have 674912 links in my database in total. Only 4009 of them is bookmarked. Some of them have tags.
I hope you liked enjoyed this information, and maybe I have inspired to also to create something new. Good luck.
Throughout my life I have found many resources worthy of my time, but which I have lost, or forgotten about. I wanted to change that. I wanted to be able to look back.
I have started my project in Sept of 2023. The core functionality is: to store, browse, and search for links. Additionally I added RSS sources as sources, so every hour I ping sources to check if there is anything new, and I expand my database. Does that replace Google search? Nope. On the other hand it expanded my horizons.
I found most of RSS aggregators lacking: - some provides ability to bookmark entries, but some do not allow to insert links manually - some do not provide ability to tag links - some are on-line, and I strongly prefer local NAS web apps - most do not provide clean ability to search link database (as it may involve heavy operations)
I found Google search lacking: - 10 pages each with 10 links for a searched keywords is not enough. If I write 'covid' into google search I would be expecting millions of links in results, from variety of sources. Even Fox news, even daily mail - I have results curated by big tech, I prefer to curate output myself
What I have found about archive.org: - it is slow and unreliable - does not store RSS sources often enough - author of RSS program objected idea to make requests to archive.org, as there could be many instances of RSS programs, which could be a problem for archive.org
In the process I have realized that I have written something similar to reddit, or digg. That is only resemblance on the surface. The link in this topic is the data. I am interested in links. I would really like to be able to import data from other uses, to see what is interesting and important for them. I like the idea of fediverses, and distributed mechanisms. I think that it could also work for search results.
Can everybody benefit from the links I have stored? I do not know. I know for certain that I am interested only in some topics, and some other people might looks at the world from a different perspective, therefore my links 'could offend them'.
Every day bookmarks and yearly repos are updated.
Projects: - https://github.com/rumca-js/RSS-Link-Database - bookmarked entries - https://github.com/rumca-js/RSS-Link-Database-2023 - all captured links in 2023 - https://github.com/rumca-js/RSS-Link-Database-2022 - all captured links in 2022 - https://github.com/rumca-js/Django-rss-feed - my humble app that I am using, which captures the links. I think that the idea could be implemented in any env (php, as plugin to nextcloud etc.)
My database is getting bigger and bigger, and I found that RSS is not a best place for my data, as written in https://www.jefftk.com/p/reverse-rss-feed. Therefore commonly everyday, has its own directory, with data in it. I am using JSON for storage, or markdown for github preview.
Inspirations: https://www.tdpain.net/blog/a-year-of-reading https://zanshin.net/2022/09/11/automating-a-reading-list/ https://dkb.io/post/google-search-is-dying https://www.youtube.com/watch?v=N8P6MTOQlyk https://www.youtube.com/watch?v=nI3GVw2JSEI https://www.youtube.com/watch?v=NT7_SxJ3oSI
Currently I have 674912 links in my database in total. Only 4009 of them is bookmarked. Some of them have tags.
I hope you liked enjoyed this information, and maybe I have inspired to also to create something new. Good luck.