Brave Search Goggles: Alter search rankings with rules and filters

wonder how they came up with that list? That list it self is a goldmine on it's own.

[0] https://github.com/brave/adblock-rust

Would be great to complement with social media as we’ll. For example, make the domains in posts within my networks, more relevant.

That way, through controlling/curating my networks, I get to decide what’s more relevant to me in search results.

Would be awesome to add stackoverflow in there too :)

I later learned that it's actually against Microsoft's Bing Search API use and display requirements (https://docs.microsoft.com/en-us/bing/search-apis/bing-web-s...), which states that you're not allowed to:

> Modify the results content (other than to reformat them in a way that does not violate any other requirement), unless required by law or agreed to by Microsoft.

> Reorder, including by omission, the results displayed in an answer when an order or ranking is provided, unless required by law or agreed to by Microsoft.

> Display content that was not included within any part of a response in a way that would lead a user to believe that content is part of the response.

And there are many other restrictions imposed upon you. I'd argue the reason you see almost zero innovation in this space (the only thing I can think of is !bangs) is because non-Google/Bing(/Brave) Search Engines have very little control over the received data, and with the little control we do have then there are many restrictions that prevents us from making these 'innovative' features.

Based on my pessimistic reading then we're not allowed under any circumstances to merge API results from multiple search engines (e.g. Bing + Yandex + Google), and for every site we want to exclude/boost/derank/rewrite sites/urls then we need explicit permission from a Microsoft representative (which would prevent us from implementing this kind of feature). We're also not allowed to display third-party ads on any pages that displays part of the API results (which is funny because I feel it's near impossible to join Microsoft's Advertising network even though my site in my opinion have a good purpose and provide a high quality UI/UX experience - https://ask.moe).. I've later switched to Google's Programmable Search Engine and even though my site has been approved by Google as a non-profit, as well as approved for AdSense, then they will only display ads in the search results if I turn off revenue-share.

This industry desperately needs legislation that makes it possible for everybody to obtain access to Google/Microsoft's crawled data, and this access should not be limited to expensive APIs that make it impossible for anybody besides Google/Microsoft to do any sort of innovation.

I have no idea why they aren't able to flag and block spam domains anymore.

Ranking is already treated as predicting which of the search matches the user is most likely to want to see. Clickbait gets highly ranked because users do in fact click on it. Goggles (which sound like a great idea) apparently let you somewhat control the prior probability distribution as an input to the ranking algorithm.

I wonder what features it lets you select on. E.g. we had a thread here recently about finding useful product reviews. It would be great to have a goggle that downranks any site containing amazon affiliate links, since those are almost always basically shill sites despite being "reviews".

Additionally we hide information behind proprietary platforms like Discord that cannot be crawled.

Ironically Googles research of text AI undermines their search algorithm because now spammers use said AIs to get good page ratings. A lot of sites have ever the same content with a few different words here and there. Especially new media content get immediately swamped with such sites since there are no established sites under certain key words. A domain filter sadly won't work as well since there a so many scamming sites.

They used to ask some users for feedback back in the good days: https://searchengineland.com/google-feedback-experiment-whic...

This is what I see: https://imgur.com/onlrA5I

Goggles allows to specify thousands of rules, not only on domain but also URL patterns and in the future matching on elements of the page as well (e.g. titles, etc.)

It is also not clear that Lenses predates the Goggles whitepaper. We’ve been playing with the idea of Goggles for a long time.

Disclaimer: I work at Brave.

Goggles white-paper was released more than a year ago, long before Kagi was even announced to the public.

Additionally, before Brave acquired Tailcat (Jan 2021) I had the pleasure to share the draft of the paper with Kagi's founder.

So no, there is no prior art.

Let me add that I do not claim that Goggles is prior art of Lenses either.

One of the key features of Goggles design is that the instructions, rules and filters are open and URL accessible.

A Goggle is not so much a personal preference configuration, but a way to collaborative come up with shareable and expandable search re-rankers.

Very different goals if you ask me. Of course, Goggles can be used for personal preferences exclusively, but that's not the use case we had in mind.

I hadn't heard of Kagi before until recently, and only just seen this Brave Goggle feature, but I had a very similar idea with my own search https://namusearch.com/ - Search using user defined url lists which can be publicly shared

I think I initially had the idea by thinking about uBlock but for Search.

Goggles have a lot of promise. I'm excited about the concepts Brave is bringing forth.

https://search.brave.com/goggles?q=long+tail&source=web&gogg...

I use the uBlacklist addon for that

Goggles is also open-source, the main difference of the approach is that Goggles collaborates with Brave Search index while your approach uses any search index as source of URLs.

That difference is fundamental, a client can use a host search engine to do query expansions to build a large result set and then apply the filters and boosts that the user defines. However, that recall set is going to be in the order of hundreds URLs (more will take either too long or the client will be blocked); and I assume it would be challenging to apply thousands of filters at once. The smaller the result set, the smaller is the effect or benefit of the user-defined rules.

Goggles, because it collaborates with the host search engine — as of today only Brave search — can apply the filters and boosts to a recall set of tens of thousands of URLs. So the net effect of such rules is much larger.

Goggles is a bit more complicated, but it's for a reason.

Disclaimer: I work at Brave.

In this case it is needed because there is a generic '$discard' rule in the Goggle (which means: discard any result that does not match any other instruction from the Goggle; you can see this as a 'default action' applied to results if they are not caught by any other instruction).

Using a 'boost=1' allows you to keep some sites, that you don't necessarily want to boost more than their "natural ranking".

We have a bit more info about that in the "Getting Started" guide here: https://github.com/brave/goggles-quickstart/blob/main/gettin...

I hope that helps!

The idea of having a separate Goggles aggregator was also mentioned yesterday; imagine a site when you can click a few check-boxes of Goggles you like, then get a new link with the combination of all of them (you could then submit this link in Brave Search to use the aggregated Goggle).

In any case, applying multiple Goggles at once is something we have discussed internally, we're not yet planning on adding this but feedback like yours is super valuable for us to decide on the next steps.

[1] https://github.com/brave/goggles-quickstart/blob/main/faq.md...

It is true that there are currently less examples of negative boosts, but it is supported by Goggles thanks to the 'downrank' option.

For example:

$downrank,site=example.com

If you want to be stricter, you can also discard sites completely of course:

$discard,site=example.com

I hope that helps,

So it's doubly punny.

No, it was an accident.

Does it put me in a bubble? Yeah. But it's a bubble of my own design, and it's a pretty nice place from my POV, that reflects my real world interests, hobbies, etc.

I see goggles as a parallel system for "I'm looking for something specific" search.

I think Goggles might turn out to be a mild retardant to toxic bubble formation, because toxic content is rocket fuel for the kind of "engagement" metrics that google/youtube/twitter/fb have spent the last decade min-maxing.

IMO, existing "search" players (Google, FB, Twitter) are culpable for toxic echo chamber formation, only to the extent that they push toxic content "by default". By allowing users to fine-tune their weights, Brave+Goggles is attempting to dislodge this norm by introducing user feedback into the equation. Like you, I think it's a good idea.

Some people will always opt-in to toxic bubbles, but I think that's more of a human/society problem and not one for a search player to solve.

Everyone has context bubbles already. You have work, outside of work, your hobbies both online(IE HN) and offline. You influence these bubbles composition. Hobbies in particular are entirely self selected.

But you are aware of which context you are communing in and are free to move between them and outside them into the general public sphere.

I see this as more of the same. As you say, "its at least transparent, people are aware of it or have to opt in"

Interestingly by the comments here, a big driver for these self selected bubbles are so that people can avoid the advertising they don't wish to see. IE Pinterest and other SEO spam. If "self selected bubbles" are bad, then "targeted relevant advertising" is worse.

The solution put forth by Goggles is to make such biases explicit and transparent, and to make them switchable. No one can honestly claim to be objective if they are using the "MAGA" Goggle. That said, people are particularly good and wearing biased goggles and believing they aren't wearing any goggles at all, or that they are wearing "the one true" goggle.

Anyway, read the white paper, which discusses your point. https://brave.com/static-assets/files/goggles.pdf

The obvious questions of morality aside, my perception of those stories is that most of the victims are hopelessly addicted to the feeling being righteous and correct and part of something bigger to the point that it takes over their whole lives. They end up a husk of a person all for a fake cause. But what is interesting is that after you take it away they generally don't find something else to latch onto, they slowly go back to having hobbies and normal conversations and normal relationships with their family, friends, and coworkers.

This is of course all anecdata, but if Goggles are another tool for giving people their loved ones back I think that's worth weighing as part of the equation.

The article also mentions that Goggles will not stop polarization, it suffices to not exacerbate it.

No technology/system on any period of time has been able to suppress it, censorship included.

Disclaimer: I work at Brave search

Allowing the users to choose their own filters will allow advertisers to actually read the market based on the the sites that are whitelisted in the filters instead of shotgunning ads at any website that claims to be relevant to the target demographic they can actually see the popular ones that users choose based on these filters.

its still targeted advertising, but abstracted one layer away from the actual user so that the targeted ads don't have to be as fucking creepy with all the data they're gathering on people. With the customer choosing what sites they want to see results from, the advertisers can stop wasting money on ad revenue for click farms that everyone hates.

its a better deal for advertisers, and provides a better end result for the user and some degree of transparency.

It can't make people accept inconvenient/uncomfortable facts, it won't solve any political problems. You can lead a horse to water but can't make it drink, you can point out any number of problems to a person but you cannot make them care.

edit- relevant to solso's comment about an active choice

The active choice democratizes the ad market allowing users to choose, instead of the passive route of allowing an algorithm to coerce the market.