Ask HN: Ways to discover stuff online without a search engine | Hacker News Reader

Ask HN: Ways to discover stuff online without a search engine

4 pointslasagna_coder25 comments

What are some tools or methods (whether real, theoretical, academic, or otherwise) for discovering things on the Internet, as an alternative to using a search engine. I'm not focusing on practicality here, just looking some interesting and uncommon methods for getting novel results. I suspect there's a few specific to certain types of media.

The methods I'm aware of are:

- Recommendation engines (e.g. Youtube, most social media feeds)

- Organic / constructed sequences of data (e.g. a playlist, or sorted lists using some measurement like price)

- Non-text search (e.g. search via image, sound, or geolocation)

- Stochastic (e.g. guessing a domain name, or entering some random text into a search engine and skipping 100 first suggestions)

- Automatic link traversal (e.g. a web crawler algorithm)

- Private sharing (e.g. sending a link in an email)

- Public sharing (e.g. web portal, or blogroll)

- Hybrid (e.g. algorithmic search engine results, news aggregators)

But what am I missing?

At least two of your suggestions include a search engine. What I can add are: word of mouth (things shared by people), news feed (someone shared something on FB/TW), ads (random ad on a random website), agrregators (specialized website for curating information on a certain topic), forum (get an answer from a random person in an internet form like HN or SO).

check out the bibliography section / resources used for a project/paper/work (on-line & offline)

univeristies publically publish thesis papers / technical reports / project documentation. A bibliography is a required feature of published stuff!

also magazines / trade organizations (ieee, acm, etc)

Ah yeah that's a good one too. Funny, one could say that this is a good way of finding interesting information, while also finding methods of finding interesting information.

neat when find a reference to things related/outside of project scope.

more detailed, but following the bibliography chain also can give the equivalent of a change log history. with the idea of, "hey, noone took this fork in the road because <insert issue(s) at time of publication>. currently though, the <issues aka missing/way to complex at the time/to costly/more supporting research needed> are reasonable/not unsurmountable."

lot of thesis papers / projects are "proof of concept"

Changelog is a good analogy. It would be nice if crawling, building a graph, and extracting some information was more straight forward and didn't involve so many paid platforms, PDFs, varying reference standards, with also a fairly high dependency on a search engine to find referenced papers. It's still not quite so easy to just "click through" and "Ctrl + F".

search engines are about as easy as it gets, at least as far as searching for available things. searching on library of congress id for published works about the only thing worse than a google site dump. certainly a step up from library index cards, microfiche, and/or writing a snail mail letter to a university department for list of deparment thesis papers.

library congress on-line stuff : https://eresources.loc.gov https://eresources.loc.gov/search~S9/m?SEARCH=Dissertation

"normalize" electronic data : https://sindice.com

It's interesting how many discovery practices rely on asking another person. That has got me thinking about the process of inquiry / learning. Another facet I guess is using some combination of tools, but developing a "method", similar to an experiment, where I have a goal/hypothesis/measurement and test some input against its output, see how close it brought me to the goal, assign some kind of value, then iterate until I've developed a kind of manual/automatic algorithm that can be applied.

I also think the idea of 2D and 3D virtual spatial positioning of links/abstracts is an interesting idea, but I can't recall anywhere I've found that done in a way I was impressed by.

historically, things were by word of mouth. internet & ease of communication have spead up the 'word of mouth'

True, I feel like automated systems also seem to still be pretty far behind in terms of providing useful results for certain forms of discovery, but is still pretty easy for people. Take this question and most of Ask HN, there's some pretty interesting/useful results within a few minutes. But providing a question like mine to google and most results will yield pages of "alternative search engines to google" websites, which is no where near correct, but does "sound" correct if you didn't understand the context of my question, such as, what the Internet is and what a search engine is (and isn't).

But then, I feel like it doesn't really scale super well, and doesn't always handle vagueness. For example, even of HN I doubt there will be very many quality/useful answers, simply because there's a certain kind of popularity required to get that here. Looking at something like Quora, there's too much room for non-answers and what amounts to spam. Then, there's the difficulty of being specific enough that people can answer but also do so in a timely manner (the speed of a search engine to provide simply a "close enough" result is part of its utility).

Like the bibliography suggestion, although I said practicality doesn't matter, its an example of how interesting stuff can be discovered (without asking anyone) but requires a huge amount of time (after the initial "look up") and cognitive effort (but that's a kind of curve that tapers off once one is familiar with a particular area of research).

well, with some prior bibliography resource gather/storing (use just pdf files, auto-pull cited references ("lines with [0-9]+") can auto prune/direct the stocastic google method noted at start.

alternatively prine method: look up authors from a set of starter bibliographs & keep authors with higher sitations/lower citations.

can use google to find number of times web resource linked/cited & use the count as a pruning method.

This sounds like a fun experiment. I wonder if anyone has already done something like what you describe.

Would be nice for the output to be a kind of collage of abstracts. Maybe convert them to audio using one of these magic AI voice synthesis things and spatially position them in a virtual lecture hall - walk past a door, hear the abstract, walk inside to get the full paper. There could be hallways of doors that are grouped together by some kind of metric.

not abstract specific, but sounds like pre-internet days promise of pearl.

sounds like an abstract doom game (pun intended), instead of sysadmin doom, where "surrounding sound would stand in for graph of connected issues.

? customize sound/room features like command line shell to tagging things of interest and/or "group of images" in room related to abstract.

sysadmindoom stuff: http://psdoom.sourceforge.net https://kubernetes.io/blog/2020/01/22/kubeinvaders-gamified-... https://eric-jadi.medium.com/minecraft-as-a-k8s-admin-tool-c...

perhaps serve as a gateway/motivation on the promise of ar.

ummmm.... pehaps doom as a pit ball of ideas,

pick/put ball in bins/on wall to make the room/refine search.

toss the idea ball(s) around, perhaps through a portal to someone else's room of doom.

join up with others to vr kick some ideas around.

well, even mashing a few software ideas together, still need to select metric(s)/subject mater to use. Although, suppose one could use some form(s) of AI to "learn" one's preferences and scan web accordingly.

"AR rooms" for front end: http://cve.sourceforge.net/

Suggested "papers" placed in the relevant CVE building/room?

Associated authors per topic:https://www.semanticscholar.org/paper/Similarity-measures-fo...

geographic association: https://www.semanticscholar.org/paper/Geovisualization-of-kn...

citation context: https://dl.acm.org/doi/10.1145/3240117.3240123

Information exists somewhere. Finding it means walking to it, checking its value for the current query and selecting that above the threshold. Walking and filtering is done either by people, or by machines or by a hybrid model. Your question excludes machines, so what we are left is asking people or finding where people have already answered the questions we might have.

My question doesn't exclude machines, at least I didn't intend to exclude them.

somewhat topic/project specific: makezine / instructables

personal web page(s) of project/thesis author(s)

on-line courses typically list resources (used, additional, related)

looking at project documentation for things on a site such as thingiverse / instructables of references to other things aka what did to be able to enable project to be done. hopefully not a wrote own os.

I feel like this begins to blur the lines between discovery tool and resource to be discovered.

The idea of instructable /project blog is interesting as a discovery tool though. It makes me wonder if the act of creating one is also a discovery tool, similar to a lecture pad or research notebook. It's kind of like, active vs passive discovery, kind of like I wrote about in the other comment, where one might have to develop a certain "method" using various tools to create a reliable + repeatable process. Once I start searching for an answer or move towards a goal, by keeping a log of the steps, the "discovery" of something "novel" might be just where the knowledge gaps are - knowing what I don't know. But like asking questions of other people, this isn't automated, and requires skill/practice/time.

even a simplified version of snarfing bibliograph data for setting up web "automata" munging with available tools, such as https://ahrefs.com/blog/link-building-tools/ , requires some skill/practice/time before "return key" worthy.

Ah I need to learn more about this.

Web directories come to mind. You didn't need to know what to look for, instead you were able to browse a huge tree. Are there any modern web directories out there?

I think mostly managed by institutions. Don't know of any surrounding niches, or managed by independent people / organisations. But also this is kind of similar to a more structured web portal.

docear : https://docear.org/ (comparison page with similar software )

** recommender system conferences:

recsys -> https://dl.acm.org/doi/proceedings/10.1145/3240323

umap -> https://dl.acm.org/doi/proceedings/10.1145/3450613

sigweb -> https://www.sigweb.org/conferences/acm-sigweb-conferences

** "Recommender Systems Handbook" with source code : https://link.springer.com/book/10.1007/978-0-387-85820-3

TinyGem is built exactly for this purpose.

It automates the process of discovery based on previous interest.

https://tinygem.org

bit more practical than https://www.semanticscholar.org/ & beach boys "AI good citations" review.

https://www.semanticscholar.org/paper/CiteSeer%3A-an-automat...

https://www.semanticscholar.org/paper/Emerging-Challenges-fo...

https://www.semanticscholar.org/paper/Deep-learning-in-citat...

https://www.semanticscholar.org/paper/Towards-reproducibilit...

https://www.semanticscholar.org/paper/Scienstein-%3A-A-Resea...

https://www.semanticscholar.org/paper/Similarity-measures-fo...

https://www.semanticscholar.org/paper/Semantic-audio-content...

Reply on news.ycombinator.com