Search engine startup from former Google Ads boss raises $40M

I would argue that a search engine can't be truly privacy focused without independence. Mojeek is one of the only ones that falls into this category. Compare their policy with Neeva's - https://www.mojeek.com/about/privacy/

First, thank you for this feedback and everyone else's in the thread, as well. We are listening and improving based on what we hear from our early users and all those interested in our work.

For context on some of the choices we made when thinking about building a private search engine:

- First, one of the biggest risks to people’s privacy online is advertising. To present a true alternative requires an alternative source of revenue than advertisers. By being a subscription service we only put users first.

- Second, a search engine that stores nothing can’t get better. Even on this thread people have preferences like wanting to never see Pinterest results or see more of them. We want to offer a better search experience and privacy so you don’t have to choose one or the other.

Unfortunately, we can’t build all things ourselves to get there. Whenever we work with a partner, we are very careful to ensure that they see only as little as they need to see. For example, while we do use Stripe to process credit card payments, they are never going to see what anyone searched on Neeva. However, in order to cover the full, diverse spectrum of what people search we have partners like Yelp, Xignite and Bing in addition to our own tech. Given what’s required, there isn’t a viable, private search engine that exists without such partners, as well. Whether others disclose them is another question. Particularly the advertising partners of the “free” search engines...

In each of these cases, we vet the privacy standards of these businesses and make sure they align with our privacy policy and values, as well. In cases like Bing we take proactive measures to protect user privacy by obfuscating or omitting all other personal information outside of a query.

Therefore, clauses like the above are meant to help bring transparency more than anything.

Lastly, in terms of aggregated and de-identified information, we’ll take another look at this language. We've already updated our policy based on user feedback and we have no intention of ever sharing de-identified information like the above. Thanks for the callout.

Keep the feedback coming! We're listening. This is hard stuff and we appreciate all the support we receive.

Do you normally buy the branded dealer part, or a generic? That would impact which ad you want to see/click.

Will you be wanting to buy this carburettor outright, or would you prefer to pay $1 per month for the next 20 years for it?

Will you also be wanting fitting instructions, a guarantee, tools etc. etc.

The answers dramatically impact what kind of products you might buy. Yet there are only slots to show a few companies/services, so unless Google correctly guesses, you will think the results are useless. They need data to make the right guesses.

People don't usually search for "groceries around me" more than once, and if you don't happen to ever drive by the storefront, you'll never know it existed.

Sure there are other ways to advertise, but they are going to be less effective than what Google can do today

It's incredibly frustrating when the front door to the internet doesn't actually do the one thing it's supposed to do.

I did a few tests. It's not as fast, it's definitely not as good for current affairs (ie, very recent stuff) and the author says it's not so good with synonyms. But outside of that it was hard to fault, so as DDG (well bing's) results are annoyingly uneven I switched to using it as my default. I didn't realise how lazy and dependent I've become on Google and Bing's ability to guess what I meant from the spelling mistake ridden crap I entered. If I put those quibbles aside, I'd rate it better than DDG. Which I find extraordinary.

For now it remains my default.

If you held a gun to my head and told me to build a Google competitor, I'd start off with smaller products with shared search infrastructure.

Start off with some hyper-focused projects that you can actually compete with Google for: code search, product search, news article search, review search, search focusing on kids, etc. Build a better YouTube search and pay a legal team to force Google to let you have some access to it. Maybe a better and free "search my site" functionality. Make some of them good enough the people just use them, get Bing or DDG interested in supplimenting their results with yours.

Have these as separate brands that people know, but for every search have a button that takes you to the wider search and start building brand recognition.

I don't think they believe they'll surpass Google. But if they can get 0.05% of Google's users to pay them $5/month, they're making $120 million a year. That's a great business by most normal standards.

(Now, I don't think this will happen, necessarily... but I understand why you'd make that bet)

Partly because google results aren't great at the moment and haven't been great for quite a while and partly because the internet is more balkanised than it was, and often I'd be happy to keep my queries within one of the partitions.

For example, search targetting just the websites that don't sell my data to 300 third parties, or that are available worldwide (not just outside the eu) or search that targets just a curated set of websites that are making an effort to provide authoritative content (e.g. wikipedia, stack overflow, respected news sources, actual academic papers). For searching for things I've recently discovered, search that spiders from aggregators and social media (reddit, hackernews, twitter, facebook) would be most valuable. Search inside audio (podcasts), video and images could also be much better.

If I could bring myself to trust them, I'd want something where every site I visited was spidered and I could do full text search within my own history.

The long tail stuff is hard, but I think it'd be OK to delegate that stuff to currently existing engines, you could provide massive value just by doing a better job of the basics.

Point being, I think the space has quite a lot of potential for disruption despite the challenges.

Infrastructure is much cheaper today than it was in 1998. Anybody with the tech know-how can start a mini search engine from their bedroom today without breaking the bank.

I see Neeva as the search-equivalent of Hey. Both products compete against a free product that Google dominates in, with a subscription alternative. Time will tell if there really is a market for this model.

Many of the constraints that they had to engineer around don't exist anymore. The landscape has changed significantly since 1998. The decrease in bandwidth costs, increase in computing power, RAM, infrastructure as a service, and the rapidly moving ML research that pops out new NLP techniques.

You also have to consider Paul Graham's advice on working backwards. Google is nowhere near the final form of information retrieval. In an ideal world, I would have an answer to a search query instantaneously. Even we solve that problem, then you begin to try to solve the problem of why you even had to query for information in the first place.

I think the index is not the biggest challenge. Yes, the internet was small when Google started and is enormous now, but there has also been a lot of concentration in the sense that the most requested content is hosted on fewer and fewer sites. Also, the expectations changed. In the beginning, the idea was to index the whole web, and search engines bragged about the size of their index. Nowadays, it's all quality over quantity, and index size is probably not the most critical factor.

Apart from that: Building a decently sized index is a major undertaking, but it is not impossible. While the web has grown, technology also has advanced, and there are multiple companies that actively maintain a search engine scale index. SEO companies, like ahrefs or majestic, do it for their purposes, and there is also Common Crawl.

The real challenge, I believe, is that Google has access to a ton of engagement data. Google Analytics, AdSense, Google Fonts, Google Public DNS, and probably other services provide Google with excellent and hard to manipulate metrics about sites' popularity.

To successfully compete with Google, a startup would have to find an ethical alternative for this. I believe that building on unethically sourced engagement data from Browsers and Browser extensions will backfire in the long run. The Brave search engine is an example of this. The best chance to succeed here have the large CDN companies, like Cloudflare, because they could harvest some of the data in a non-invading way.

> [..] and let's face it, in the English language, it's pretty much all Google with Bing a distant second

I'd argue that the non-English world is, even more, all Google because it is the only search engine that works appropriately for non-English queries as well as for local search results. Baidu in China is the only real exception to this. Even in Russian-speaking countries, Google's market share is higher than their (decent) local search engine Yandex.

For a general search engine yes. Less so for a specialty one. If I'm searching for javascript documentation I don't care if it's attached to a domain attached to a business with a physical location near me.

Imo if a search startup appears it'll be through a tiny niche -> larger niche -> even larger niche -> general search, path. Not nothing -> general search.

But you're dealing with insanely long-tailed distributions and the "meat" of the search engine business is in the fat heads of those distributions.

(1) A small proportion of queries makes up a huge proportion of query events that you'll see through the day. (2) For any given query, a small proportion of users will make up a huge proportion of the opportunity to monetize (researching a planned purchase, looking for a job etc). (3) For any given query, an infinitesimally tiny proportion of the documents on the web is where the value to the user actually is.

I think there are potentially many ways of making selections on each of those three axes and ending up with a viable business based on a manageably small search index E.g. indeed as a job search engine or amazon as a product search engine have manageably small document collections, great value to users and a stable user base, and great opportunities to monetize.

...from that point of view I find it surprising that there aren't a lot more search engine businesses.

Case in point: I really like wiby.me, the 90s nostaliga search engine. I don't think they are a profitable business, but I certainly think they could & should be.

And even Google is making very deliberate choices along those three dimensions, rather than naively indexing the web and naively executing keyword search against that index.

Dimension 1: Given a query, reinterpret as follows. Catch eyeballs by delivering entertainment value ("pizza" -> "entertaining videos related to pizza"). Monetize those eyeballs by reinterpreting as local queries ("pizza" -> "restaurants near me wanting to sell me pizza") or products queries ("pizza" -> "online shops trying to sell me pizza ovens or other pizza-related items")

Dimension 2: Given a query and document, always make relevance decisions on behalf of the kind of audience with disposable income. E.g. "farming" shows a lot of stuff you'd want to read if you're the kind of person paying £10 for a potato in Borough Market and nothing that you'd want to read if you're a subsistence farmer in Namibia.

Dimension 3: When comparing two documents for inclusion in the pool of documents that stand any chance of coming up on page #1, prefer recency over authoritativeness. E.g. for "programming languages", apparently "The 9 Best Programming Languages to Learn in 2021" is considered relevant while "Go To Statement Considered Harmful" or the Mozilla Developer Network are considered irrelevant.

There are huge audiences who don't agree with these choices that Google is making and who are just waiting to switch if someone comes along making better choices or using another product alongside Google if someone comes along simply just making different choices that are useful in some way.

So, again, far from the sentiment that any venture in the web search space is doomed given the size of the web and the existence of Google, I'm positively baffled by the fact that there isn't a lot more activity there.

> In 1994 — the same year the Highlands Forum was founded under the stewardship of the Office of the Secretary of Defense, the ONA, and DARPA — two young PhD students at Stanford University, Sergey Brin and Larry Page, made their breakthrough on the first automated web crawling and page ranking application. That application remains the core component of what eventually became Google’s search service. Brin and Page had performed their work with funding from the Digital Library Initiative (DLI), a multi-agency programme of the National Science Foundation (NSF), NASA and DARPA.

> But that’s just one side of the story.

> Throughout the development of the search engine, Sergey Brin reported regularly and directly to two people who were not Stanford faculty at all: Dr. Bhavani Thuraisingham and Dr. Rick Steinheiser. Both were representatives of a sensitive US intelligence community research programme on information security and data-mining.

> Thuraisingham is currently the Louis A. Beecherl distinguished professor and executive director of the Cyber Security Research Institute at the University of Texas, Dallas, and a sought-after expert on data-mining, data management and information security issues. But in the 1990s, she worked for the MITRE Corp., a leading US defense contractor, where she managed the Massive Digital Data Systems initiative, a project sponsored by the NSA, CIA, and the Director of Central Intelligence, to foster innovative research in information technology.

>...

> Thuraisingham goes on in her article to reiterate that this joint CIA-NSA program partly funded Sergey Brin to develop the core of Google, through a grant to Stanford managed by Brin’s supervisor Prof. Jeffrey D. Ullman:

> “In fact, the Google founder Mr. Sergey Brin was partly funded by this program while he was a PhD student at Stanford. He together with his advisor Prof. Jeffrey Ullman and my colleague at MITRE, Dr. Chris Clifton [Mitre’s chief scientist in IT], developed the Query Flocks System which produced solutions for mining large amounts of data stored in databases. I remember visiting Stanford with Dr. Rick Steinheiser from the Intelligence Community and Mr. Brin would rush in on roller blades, give his presentation and rush out. In fact the last time we met in September 1998, Mr. Brin demonstrated to us his search engine which became Google soon after.”

_____

Anyhow, I recommend everyone to read the full piece and it's sister piece where it goes deeper into NSA things

https://medium.com/insurge-intelligence/how-the-cia-made-goo...

https://www.surveymonkey.com/r/neeva

That alone is not very comforting.

Although afaik google too does not sell profiling data, rather they sell access to highly speficic demographics.

Any true Google competition will need a world class crawler and ML like Google.

I'm sure this is not the standard sort of "entrepeneur under the VC's foot" type of situation.

Do you mind clarifying what you mean by "being tracked" here ? Regardless of exactly what you mean, if tracking is the only criteria, what do you think you'd gain by this "paid search that's not tracking" vs using DuckDuckGo or using Google with a fresh incognito window ?

I think they should tab their results: Information, News, Shopping