Now you can search on Google for free: Solution with API | Hacker News Reader

Now you can search on Google for free: Solution with API

github.com

33 pointskarust24 comments

Why is this valuable, who is it valuable to?

If you're just looking for a way to scrape search results, what end use are you trying to serve with that?

I'm not saying it isn't valuable, but as maybe the tagline needs maybe needs work. As far as I am aware, I am already searching google for "free".

It is my understanding that search engines heavily use cookies in order to tailor results to the user.

Use an LLM to go through the first 50 links, explore the content, and give you a summary of your query along with the link to the best result.

The search is indeed free. But if you try to automate the process, it becomes "paid" :) For instance, I use it as part of my other solution. To inspect the content of URLs I find during some queries.

It's a wrapper around a headless chrome apparently, fyi. This will see some growth is my guess. I'm also working on something like that with 2markdown.com. A pity that you need a full blown browser to retrieve web content. Maybe these weird semantic web folks have been right all along ;)

Won't you get served captchas if you're querying frequently?

I doubt it. Its user agent should look like a normal Chromium browser, and under a normal residential IP it should look like normal usage. Under a proxy service or VPN, however, that may be different.

the amount of usage per IP-address is limited though

Yes.

Actually depends. I prefer rod to drive headless chrome and you can use rod-stealth for that. You have to mimic traffic to replicate humans. And if you don't spam a website it won't trigger any defence mechanisms.

Check https://bot.sannysoft.com/ ;)

Semantic web wont do you good. Google doesn't like scraping and won't make it easy for you to do so.

Lately I've been struggling to get search engine results via API. If you ever had the same situation, this project might help you

Please tell me you aren't making the same mistake of building any products around this. I can't stomach another startup "we're shutting down" Tell HN post.

>API access for search engines results if available isn't free.

Ahah, no, that's mostly for small researchers like me. Creating a startup around it would be difficult, with questionable financial capabilities.

Now do Reddit next :-P

You can just extract the API key from the official app and patch your third party app of choice to use it, at least on Android.

What if the app is closed source

Doesn't matter, you can patch binaries as well. Here's patches for Boost and Sync for example: https://revanced.app/patches?s=reddit

quite similar to SearX, but I don't think SearX uses headless chrome. https://searx.github.io/searx/

Yea, they don't use anything that heavy. I use https://github.com/searxng as my primary search engine (fork of searx)

I also use searxng as my primary search engine protected by Authelia on a remote server. This way, the IP logged is my throwaway homelab using certs from Let’s encrypt. It’s pretty nice overall and takes not too much RAM (350mb?).

Are they scraping the Google web interface or using the API from Google?

scraping - just looked at the code.

As far as I can see reverse image search is not supported?

Not yet, I have to do some research with this one

Reply on news.ycombinator.com