Show HN: PyTorch search engine | Hacker News Reader

Show HN: PyTorch search engine

you.com

65 pointschris_f17 comments

If you could search through github projects (torchvision, mmdetection, etc) and index high quality pytorch snippets, that would be awesome. I learn a lot about how to manipulate tensors and achieving specific deep learning tasks from GitHub projects, that’s how I answer various questions on how to implement key ideas with pytorch.

Good stuff! Github is one of the sources, but not specific repos. I actually think we can break out Github into individual repo sources pretty easily.

Niche search is a huge problem with Google and other big players these days, happy someone’s facing this issue head on.

Would be great if you could do this for Web3 / crypto, and other such niches where tons of new info is coming in and poorly indexed

Any specific sites? Happy to spin one of these up for you focused on Web3. These work the best when the search engine creator has domain expertise to showcase the best sources.

I could try to find some sources, but my guess at the best Web3 sites would probably miss the mark.

What kind of info do you need to find on the crypto space that can’t be easily found with Thegraph or similar tools?

Here are some more example queries:

https://you.com/niche/pytorch?q=install+osx&fromSearchBar=tr...

https://you.com/niche/pytorch?q=save+trained+model

https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GP...

For the first example the snippet isn't correct, it just says `conda list`. The website (TutorialPoint) it links to is also wrong and useless.

Appreciate the feedback, it is helpful! The relevance tuning is a continuous work in progress.

Sounds like a good idea, but I'd like to see how is it better than googling these phrases together with pytorch keyword. Not saying it's not better, just not clear if it is.

In the broadest sense, a highly targeted search engine (like this) can provide better results because Google has to determine user search intent AND return the right results from trillions of webpages. The advantage of this search engine is that all of the users are looking for the same type of information, and the result sources can be curated to ensure high quality and relevant results.

The more targeted the topic, the harder the time Google has to provide quality signal through the noise of SEO and sheer volume of content on the web.

In addition to the above, the UI provides some cool features like allowing horizontal scrolling of sources to provider higher information density (important for discovery), and some source content can be viewed in the side pane without leaving the page.

But ultimately it would be good to hear if this approach does make it easier to find relevant and higher quality Pytorch info.

I just tried searching for: pytorch Distributed model parallel, and google fails completely to return relevant results in the first few pages, even if I put Distributed model parallel in quotes. You.com also mostly fails, but it returns a section specifically from Discuss Pytorch forums which does have relevant results, so yes, this is a good sign.

That's great! And I think the results even improve a little more if you just search 'distributed model parallel' without 'pytorch' [0]

So still some work to do, but I've personally found this type of search engine really valuable for some of my interests where I am able to curate the sites that might not normally rank, but I know are higher quality.

[0] https://you.com/niche/pytorch?q=distributed+model+parallel

This is great. However how to find other such niche searches. A request for you.com/niche returns a 404 page which means you cannot find other niche searches like this one

Yeah, that's a good point. As new ones get built, we will need a way to make them discoverable.

Also, just as a heads up, there is a link at the bottom of the search page [0] if anyone would like to build one of these search pages on any topic that might be interesting. It's just a form now bc we are working through the process.

[0] https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=119356944...

Thank you Chris. If I may add, maybe have an approach to get some community feedback on such niche search as well. This can help to narrow down the list. Great work. I'm going to use this search for my work

that seems like a super valuable domain being very under used.

I think (at the risk of sounding like a shill; I don't use them) you.com is an interesting search engine that is not very well known. I'd be much happier seeing it be used for that than some we.com type branding play (although I take your point that in the hands of some branding expert it could have maybe a more clear connotation than anything related to search.

Reply on news.ycombinator.com