Great job! How are the ads implemented and do they cover the costs? I'm thinking of building a similar search engine for a completely different domain. I am a bit concerned about paying for it though.
Ads are added automatically by Google. The whole thing is little more than a wrapper around the 'Programmable Search Element Control API' which is an HTML element you can just insert into any site, like an iframe. Unfortunately this is the only way to make Programmable Search available at scale as the API is restricted to either 10 sites or 10K queries / day, even when paid!
There is a paid version for the HTML plugin, but that would leak the API key and so it wouldn't work as a business.
There is an option to get a share of the revenue generated by a search engine. Maybe it's time for me to figure out how that works.
I was thinking of making a hosted, ad free, customizable version where people upload their own keys. Not sure if people would like that.
As a side-note, it's super easy to remove ads with 1 line of CSS, but I wasn't sure how Google would feel about that so it's not in the online version. TamperMonkey is an extension that allows people to insert their own CSS on different websites. Hmm.
You can view all offerings in the docs [0].
[0] https://developers.google.com/custom-search/docs/overview#su...
Right now, looking at your allow-list config, it feels a bit custom to you, but if I had an easy way to limit search to the sites I myself know and trust, I could see how that would be useful.
I know I could probably pick it out of my browser's history UI & poke it into Google's Programmable Search UI, but that seems like a hassle and a half.
* easy add to my filter list (like maybe a browser plugin so I can see that the current site isn't in my filter, but I can click a button and now it's in my filter & opposite for remove for when sites start to suck)
* stats on which sites I visit after searching
* aggregate bing+google filtered searches
* curated site lists for different topics, top 100, etc. Maybe like a temporary search using these sets such that I can try them without affecting my own filters. Maybe sharing lists w friends
* some sort of search anonymization/log deletion feature
* integration with browser search on desktop & mobile
* search flags like duckduckgo so I can easily switch filter sets by typing like /news or /nerdshit in the query
* integration with archive.ph & wayback machine
With caching, I think you might be able to reduce the load.
Also, why is w3fools in the list? It's an awful site.
If you want to be cynical, just do Bing/DDG searches over Tor, and scrape that into the cache. This is $0/1000 searches, though it obviously violates some ToS somewhere. Unless they want to block Tor, you should be good.