Hacker News Reader

Comment by seunosewa | Hacker News Reader

I loved del.icio.us. It was a core part of my browsing experience. Any useful link I stumbled upon got tagged and saved. The popular links were very useful, too. Then it got sold to people who couldn't figure out how to make money off it without ruining it.

I still miss the functionality of being able to quickly find every interesting webpage I've ever seen (using tags). A way to supply that functionality in the modern world would be a visited pages search feature on Google or Chrome. Or a search feature for the content of pages I've bookmarked.

View on HN

Pinboard (https://pinboard.in/) is still a thing, and the developer bought the deli.icio.us domain too I think.

In any case, it has at least replicated the del.icio.us functionality, and then added more, such as archiving page contents. The tags are still there too, and it prompts with other users' tags when you add a bookmark. Oh, and an API, which is very useful for programmatic use of the data once saved.

Pinboard doesn’t have the social features that del.icio.us had—you can’t see the list of others who bookmarked a link, for example.

In my opinion, social sharing usually requires free accounts. If the account costs money, then the social sharing features are probably less important. Pinboard says it is for introverts.

If proper free accounts with all functions are supported, then advertising is a likely revenue model.

To combat this problem, I've added sharing over email, SMS and Slack to my project, which is a personal search engine and document manager. There's no reason to not allow a very basic and free account function to read these shared links, without a lot of auth and registration getting in the way. The service already has the guest's email address, phone number or Slack account from the other user. This can be used to send a token for easy login. Limits would be added so these accounts can only store a few URLs themselves unless they upgrade.

This keeps privacy protected, hopefully.

Yes, you can see people who bookmarked a link (if they haven't made it private).

Wait really? How? I haven’t been able to figure out how to do this, but I’d love to know.

OK, so this seemed like something plausible to do, thought I'd take a look.

- Looked at "popular" bookmarks, e.g. https://pinboard.in/popular/ - At the far right end of each page title, there's a little number. For example, beside "Lotus 1-2-3 For Linux", there's currently the number 21, which links to https://pinboard.in/url:61d59935774d1affe22713a7423c78bd17ef.... - Following that link, you see who bookmarked that link, with name, tags, and some related other links. Of course, clicking on a user name will show you other things they've linked, at least using the default public setting.

I'm building a personal search engine/document management system that uses tags similar to how del.icio.us worked. URLs and screenshots can be saved via the browser, or by instructing the system to crawl it (which gets done with Firefox/webdriver). It's a like a split brained version of the Grub crawler. It also supports uploading PDFs and images.

Tags, objects, labels, synthesized commentary, etc. are provided by machine learning models and GPT3. Eventually the pipelines will be customizable, so running a plant identification model will be possible. Full text search and analytics is provided via a customized Solr deployment manager. I've built a unique UI for it based on my original cut of a simple timeseries interface at Loggly. Love using it, but have no idea if others will want to pay for it. I seriously hate ads, trackers and user privacy violations.

  merry-zebra|> !crawl https://news.ycombinator.com/item?id=31459103
  merry-zebra|> Please wait while I index https://news.ycombinator.com/item?id=31459103.
  merry-zebra|> Site has been indexed. An image of the site will be added in ~10 seconds.
  merry-zebra|> ...
  merry-zebra|> updated 2022-05-21T18:55:06Z
  merry-zebra|> ID UmXyyk3tZJdGZW4uv
  merry-zebra|> title What Happened to Tagging? (2019) | 
 Hacker News
  merry-zebra|> description The article discusses the potential reasons why "tagging" (i.e. adding labels to content for organizational purposes) has declined in popularity in recent years, despite its usefulness.
  merry-zebra|> URL https://news.ycombinator.com/item?id=31459103
  merry-zebra|> Tags #What, #Happened, #Tagging, #2019, #HackerNews, #News
  merry-zebra|> ...
  merry-zebra|> To search me for the document, click on one of the action links.
  system=> Do you have any comments about this webpage, @merry-zebra?
  merry-zebra|> I find tagging to be extremely useful for organizing content. I think the decline in popularity is likely due to the fact that it can be time consuming to tag everything, and people are often lazy. However, I think it is worth the effort to tag things, as it makes it much easier to find what you're looking for later on.

Is this a project you intend only for yourself? Or is it going to be a product?

It's a hosted service that will be available as well as an on premise deployment for companies.

Nice work. How do you crawl dynamic websites that barely use links, or those which have scraping countermeasures like Amazon?

Thanks! I use GPT3 to synthesize a title and description from the URL and also use it to generate a description if the site simply lacks one. I use webdriver running Firefox to image the site. Some DOM information can be pulled that typically isn't blocked, but it isn't implemented yet.

My argument for these companies to allow a "scraper" like mine, is that I'm adding their full URL and tags for the user, on the user's behalf. I'm not scraping URLs or doing breadth/depth crawls. I ask for a single page the user gives me, then take an image only that user can see, unless they chose to share it with someone over email or Slack.

When a site implements block "crawlers" from certain IP blocks, I've written an extension for Chrome/Firefox which allows the user to image the screen and upload it. This adds the site to the index just like if they asked the site to crawl it. I gave up on scrolling the window, however. 0.5 seconds per screen grab limit in Chrome now.

It also supports image uploads, so if the user wants to just use their own screenshotting method, they can just upload the image. Extraction of text and synthesis of titles and descriptions can be handled by GPT3 (as well as URL synthesis from keywords, command translation and Solr query synthesis).

I'm working on training a model to tell me whether or not it's an image, web page screenshot or a desktop shot.

I've built something similar for my crawler at biztoc.com — It does OpenGraph extraction, finding the body, summarizing it, tag & entity extraction, sentiment analysis, Oembed, stock symbol detection, screenshot & favicon, etc.

Great job on this! It looks fantastic and I think you'll do well. I like how you moved to token use for logins. Passwords are dumb.

I thought about these types of features for Mitta.us (which is NOT done, but operational), but it was too much work. Glad I put it off, because you did a much better job.

I'm adding a !biztoc command to Mitta for search, but it would be cool to be able to add some post parameters like https://biztoc.com/post?title=foo&url=https://zombo.com to post as well.

Nice! That's already implemented: https://biztoc.com/s/bm

E.g: https://biztoc.com/post?bms=mitta&bmu=https%3A%2F%2Fnews.yco...

Let me know if you need more. Cheers

What's the best way to reach you? I am building something with a lot of the same ideas and would love to talk shop. Trying to network with others in the collaborative search/organization/knowledge space.

My email is also in my bio.

Completely agree. Delicious was like the perfect bookmark manager. Then it went to complete shit and ever since then I’ve barely bookmarked anything.

Honestly though I don’t think bookmarks serve much of a purpose anymore. Like I’ll just search my history if I need something specific. Or maybe I‘ve just forgotten how useful they are.

Am I missing something because I use Firefox? I bookmark and tag every interesting site I come across. Is tagging not a thing in Chrome?

> A way to supply that functionality in the modern world would be a visited pages search feature on Google or Chrome

I've been wondering about a plugin that does that. Maybe built over this? https://lunrjs.com/

I am ABSOLUTELY CERTAIN this does not yet exist.

(Easier to type that last sentence than actually Google for it).

Why couldn't such a thing be a local browser extension or similar?

Reply on news.ycombinator.com