The future of search is boutique
future.a16z.com
future.a16z.com
The stated mission of a company worth almost two trillion dollars is to “organize the world’s information” and yet the Internet remains poorly organized.
The first examples are information. The last example is not really information.
I get that search is so convenient and good at the things it's good at that you want it to be good at other things, but you shouldn't expect a search engine to make judgements on your behalf or put things into your context. The searcher has to do some legwork of their own there. Maybe I'm a control freak in this regard, but I trust myself more than any website in this regard.
Wire cutter is one of the curators they bring up. I don't trust that wire cutter manually curates top products. I trust that it only curates products it can make affiliate links to. Maybe these products are good enough, but maybe not. For that reason, I don't know what the value really is, since their incentives don't necessarily line up with mine.
I found a lot more value when looking at 3D printing forum discussions from passionate hobbyists to see their opinions about the best printers on the market for various needs, compared to a search engine. In this case, the curators would be the forum moderators who would need to do the work weeding out obvious shills and misinformation. Presumably this moderation would be increasingly difficult if more people make their purchasing decisions like this and businesses respond with more astroturfing.
For something like NFT (or investing in stocks or real estate or anything), I'm not sure there will ever be a good answer for that because the super obvious conflict of interest. "It just so happens that the NFT/Stock I hold is the best investment" is not really interesting or useful and would be terribly difficult or impossible to weed out.
And we're just talking different flavors of Ponzi schemes if we're trying to design a system where available revenue doesn't cover costs.
I remember pre-Google, when it was an open question as to whether curated directories or search engines would be the dominant form. Turns out, the former when signal>>noise (early web) and the latter when signal<<noise (current web).
Reddit is a gamified answer to a fundamental imbalancing of profit: convince people to do valuable work for free, pay them in karma/gold stars/Monopoly money, and then sell that work (without having to pay its fair market value).
And for profitable search curation niches, there are already solutions. F.ex. Bloomberg makes $10B of revenue off one.
You are not. Sometimes you need a search engine, not an answer engine. Google/AI can be a force for good, but often it is not; some examples & views of others covered in a thread [0].
[0] https://twitter.com/ColinHayhurst/status/1420698145898520579
google's mission statement is not "to organize the world's facts" but maybe you'd like it to be
This is true but it Google still tries to despite the fact that they can't. If Google returned completely unbiased results and let the reader filter the results then I think people would have fewer results.
Just speaking from my experience in thinking and working on this concept over the last 2 years...I could be wrong, but testing shows some hope in this idea being a reality.
The document set is the critical part here. The user must obtain the documents that are of interest to them. Keeping the document set small and focused allows for higher quality results and discussions about the content.
And then when you need to search inside the Yellow Pages, you'll ask yourself why YP doesn't do something about the misleading descriptions provided by the companies in their index.
> NFT collectors think that NFTs are a great way to collect and trade digital assets. They also think that NFTs have the potential to revolutionize the gaming industry.
Outside of Google's attempts to summarize information in the answer box "questions" posed to a search engine are really "find me documents which discuss the subject I'm querying about and might contain my answer."
That suggests that a platform that aligns with the needs of the advertiser is likely to earn more revenue than a platform that aligns with the consumer.
For this reason I find it hard to see a dominant "boutique search" company that is funded by anything other than advertisers.
"Boutique" to me means high end tradespeople focusing on a specific point of view over growth. I can't imagine a search service fitting that category and allowing advertisements. That's not what either party involved is there for.
This is the main problem with Uber. There was never any need for investment. The service itself could run perfectly well without anyone skimming off the top, and there wouldn't be as many mouths to feed running things.
We need an open big-list-of-URLs exchange standard, then one can use some sort of app to create/rewview, upload/download and import/export lists of favorites/URLs.
I prefer the auditors skepticism; anything that can be corrupted, given enough time, will be corrupted.
Also seriously stretching the definition of a “search engine.” I certainly wouldn’t consider LinkedIn a search engine. It’s a social media site with search, and recruiters can filter their queries on people but it’s a closed system. Only LinkedIn accounts are there, unlike the web which is open and needs a true “search engine” to organize it. If the web is open, then to me LinkedIn already is curated, and then what the author is proposing is further curated still but certainly not a “search engine.”
Snark aside: The random, bold “Nascent token-based business models show early signs of promise.” from an author who has no understanding or experience with them and is tacked on at the end just makes me picture someone at corporate writing this in in sharpie.
Feels like "trust" that they're actually seeing the best results are going down.
That said, many of these same people rarely actually google things outside of simple factual information that Google does okay at (e.g. height of the eiffel tower). Their experience of the internet is mostly through various social media filters (e.g. Facebook, TikTok, Pinterest, Instagram, Twitter, Reddit, etc.)
I also suspect this is part of the reason Google is giving worse results. Most of the actual content being generated on the internet is now occurring in various walled gardens that either pollute search results (Pinterest!) or don't show up (Facebook).
It's a tough problem for a company built on the open web that web mostly resembles a late-game Risk map with just a few big players.
To 'normal' people Google is the synonym for internet search and yea you are right nobody will be breaking the habit of "just Google it" anytime soon.
Google's lack of awareness regarding the power users is astonishing but a change needs to come from within Google not from outside because Google's top management just doesn't care what power users think.
By now, Joe User has developed a healthy distrust/dislike of Facebook, but they stay there due to the network effect. Google has no such snares.
I'm pretty sure that takes a bunch of clicks and is confusing.
> this is why google pays so much to Mozilla.
Google pays Mozilla to fend off antitrust.
edit: Could Google disappear in a day? Maybe. People are still Xeroxing things but nobody is using a Xerox. Google have so much cash that they can pour into marketing that people would gradually drift back after any stumble, though. At the least they could get regulation passed that would be too expensive for newcomers to comply with, or that requires that Google be used as a middleman for some processing.
Just from performance perspective I can see that the listicles and how to's are doing better then any other niche topics.
What if there was a premium Web where each website had an ISBN-equivalent, and the dates during which it was "in print?" And to get one published you'd pay a fee to a central register, such as $100.
I guess there is no Google for books, and no ads/affiliate links. And there are libraries where books are manually curated using the Dewey hierarchy.
Listicles do show up in print though. I see them at the grocery checkout stand.
How many Kindle direct releases are going to get that kind of backing?
So far all I've seen is shittier versions with terrible UI(you.com), or exact low-risk copycats going at it with the privacy angle.
1. direct link, https://breezethat.com
2. tons of date filter examples to get newest info or to go back in time, https://twitter.com/search?q=from%3Adotdotjames%20(date%20OR...
3. while there are several experimental topics in the drops menu at top, the best examples is probably our jobs filter at 14M+ openings, https://breezethat.com/x/job-search-beta
4. gradually merging topic filters into main UI
What is often missed even by those who were around and paying attention in that context back then, is that most people were blinded by the "good enough" factor that rather mindlessly simply assumed that Google served up if not nearly, approaching perfect results ... that there was simply no better results available. A clear fallacy, as one could and especially today can prove on their own by simply taking some site and page with the most important or even niche information on a topic, and then trying to search Google for it. You will likely find that there is approaching no string of basic search terms that you can use to find that resource. Even if it is a page that is part of a major, mainstream site, it is immensely difficult to find information using general terms.
I would argue that all the censorship and corporatizing and centralizing of information and data has only amplified that problem even beyond the early days. As a life long user of Google search, even long before there were public discussions of what a google is and people still thought AOL is the internet, I feel like the peak of Google and search in general occurred some time around the mid aughts, so, 2003-2008.
I would love to hear from anyone with similar direct contextual knowledge.
> Sari Azout is a founder of startupy.world, a search engine for tech and culture insights.
However the point is, search engines were meant to scour the web for data and present them in the order the algorithm deems most relevant. That is it. Google built a vertical integration into it for specific types of data that could be turned into a single answer like bus routers, NBA scores, temperature, wiki info etc. This is not the norm. It can only be done for some data types.
"Boutique" search engine by definition is doing logic analysis based on context, and so that will bring in "someone's" point of view into it. Immediately becomes useless for objective research/analysis. This is supposed to be done by the user. IF "boutique" is needed we already have it in reddit and this forum, media outlets like tv and print.
The specific question asked in the article "What do NFT collectors think about NFTs?" is a terrible one because a search engine cannot achieve that objectively. Someone needs to conduct a poll and that too its subjective because NFT collectors for different verticals/price points might have different opinions. NFT naysayers and NFT platform owners will have different motivations too. So I don't know if we should trust a boutique search engine to tell us a one line answer.
The author also assumes must scale signal instead of reduce noise. She plays the futurism card and tries to attack the Yellow Pages. Why. The Yellow Pages has zero noise. Google has infinite noise. Gee, I wonder why. For starters, the telephone companies did not keep rearranging the alphabetical listings to some sui generis order based on some secret algorithm and create a competition to see who can be listed first, then auction off ads to be placed above the first listing.
"How many New York Times accounts did you create before finally giving in to the paywall?"
None. I do not use cookies or Javascript to read the NYT. I only use HTTP and HTML because that is all that is needed.^1 NYT chooses to share information publicly, no password is required. Why. Because if they don't someone else will. The internet allows its users to share data and information. They do not need a printing press. User publishers have already paid for that privilege when they paid for internet access.
Given that we already pay for our computers and our internet subscriptions, why do we need further intermediaries telling us what to consume, for a monthly fee. I say we don't. The time has come to stop trying to exploit people as they access public information, be it through surveillance and advertising, or "subscriptions".
The idea of asking for a subscription fee to search what is available for free over the internet is nothing more than needless "gatekeeping". In 1993, I sat down at a university computer to use gopher, ftp and the www for the first time. Nevermind not knowing what to search for, search was slow. Getting access to the internet was neither free nor easy then. It still isn't free today, but it's certainly easier. Search is much faster. The problem of not knowing what to search for is a good thing. It forces one to explore. Intellectual curiosity instead of marketing.
From what I have seen, the number of new www users is growing faster than the size of the www, but Google is unlikely to let anyone know that. Like the Wizard of Oz behind the curtain, it is better for business to keep up the illusion that the web is too large for any mere mortal to investigate without Google's help. But today we can scan the entire IPv4 address space in a matter of hours. That time will keep getting shorter. Exploring obscure corners of the www is not something only Google can do. They have no motivation to do it because unpopular websites with low traffic are not suitable for Google's programmatic advertising.
Now we have Silicon Valley VC telling that us charging a fee to conduct searches is the "future". The reason the www is filled with noise is because of VC funded "tech" companies, like Google, that only focus on the www on condition that it serve as an advertising medium and/or a means to collect user data and generate metadata about users. VC and their "tech" company spawn want monopolies, millions of people all using the same websites, so-called "scale". The possibilities for millions of small businesses, not beholden to a "tech" company providing a "platform", even just people using the internet to exchange text, images, audio and video for non-commercial purposes, without the need for an all-knowing, all-seeing intermediary spying on their every move, is not something VC are interested in. The www has suffered the consequences of VC "thinking", including endless noise as a result of their sponsorship of hype, a race to the bottom competition for lion share traffic at any cost. The future of the internet is one without the corrupting influence Silicon Valley VC. Enough of the noise. We can do better.
1. Visiting https://www.nytimes.com gives the public JSON file
https://static01.nyt.com/services/json/sectionfronts/$1/index.jsonp
Alternatively we can use sitemaps, e.g,
https://www.nytimes.com/sitemaps/new/news-$1.xml.gz
For example,
curl -A"" -4o 1.jsonp https://static01.nyt.com/services/json/sectionfronts/world/index.jsonp
exec sed '/\"guid\" :/!d;s/\",//;s/.*\"//' 1.jsonp
exec yy059 < 1.jsonp # yy059 is a quick and dirty JSON reformatter I wrote
curl -A"" https://www.nytimes.com/sitemaps/new/news-world.xml.gzOn Reddit you have a very good chance of seeing multiple high quality, well-informed posts on whatever topic you're interested in, written by real people who genuinely want to offer help in an area they're knowledgeable about.
On Google these days, even once you scroll past the wall of actual ads, you'll usually just see a bunch more defacto ads masquerading as "organic" results.
The pervasiveness of fake, shill, and/or paid reviews across the rest of the web makes it difficult for me to escape the suspicion that there is pervasive review fraud on Reddit as well.
I see this sentiment a lot. Are there any actual examples of this happening, or does it merely propagate on the basis of sounding true?
It's "manipulate the results", by way of "be careful what you wish for" - if there's money to be made in pandering to a search engines whims, then that will happen at scale and that content will swamp the good stuff that you wanted in the first place.
> When a measure becomes a target, it ceases to be a good measure.
So, when you try to measure content quality in some way, that becomes a target, and starts to distort the quality of the content.
I think this applies equally in any niche, at any scale where people find it worthwhile.
If you have a niche search engine that has different content requirements, and it becomes worthwhile to SEO niche content for that engine, then it will happen, eventually.
I think it just has to become worthwhile (and then for this fact to become known) - but that threshold is going to vary a lot, in different niches, times, places, etc...
There are plenty of people (~everyone?) gaming/SEO'ing the search on Amazon, Etsy, Ebay, etc... for example, by doing niche & platform specific SEO on their listings on those platforms.
I also think that people's definition of "worthwhile" differs too. For the broad SEO industry, it monetary. For people making posts on HN or Reddit, it's Karma, or Discussion or something - and people will optimize their content towards that, just for "fun".
Actually, I think this applies pretty boardly to making _anything_, more or less, once you have someone or something evaluating/measuring your output?
How _harmful_ this is, depends on what's being optimized for; scale & rewards probably change how "hard" the optimization arms race gets - as well as how many people are affected.
And that was when Google was over 12 years old.
There was a time I only saw listicles on Cracked.com. Now most of Page 1 content on Google is structured like that, and most are low quality blogspam.
Essentially “goggles” allow anyone to create their own algorithm and share it. It’s a really interesting concept. Definitely worth reading the paper.
Wikipedia, millions of volunteers and bloggers organise the world's information. Google just leaches off them, they cant even organise Youtube
Does youtube prioritise videos that have the most informational value, news that are most objective? No, they financially reward those that produce misinformation and conspiracy theories. You could argue 'well, thats what sells', but then you can't claim any higher purpose or mission - its just peddling whatever is popular, like fast fashion.
Or do an experiment, search for a government service like 'open a company in UK' or 'apply for a driving lisence', queries where there is only a single government service per country - and google will return a page full of scams.
During the pandemic i tried to find COVID related rules for entering moldova- the result was on like page 7. If failing to find official public information is not 'ublnderperforming', i dont know what is.
The premice of this thesis is wrong - no-one is seruously trying to make the search engine that produces the best answers because the competition is basically gone, and because the web has been balcanised from open system into feudal kindgdoms.
i love it and yeah I'm at the point now where i do want my search to be boutique. me searching for something and my grandma searching for something are two completely different animals.
(vladimir in question here)
> Someone who wants to find the best freelance designer, or the best sushi restaurant, or the best NFT to buy will not find the answer on Google.
This is a poor article IMHO. The reasoning is as follows:
1/ We are looking for things to consume, and we can only accept the very best of anything -- because we are busy (and, let's face it, exceptional geniuses, contrary to people from a decade ago who had all the time in the world and were far less bright);
2/ Generalist search engines can't help us, because they lack category knowledge necessary to "rank" things
3/ Only curators can help us get there.
None of those points are obvious, and they don't follow from one another. Not everyone is looking for things to consume, all of the time. We don't need the very best of everything. There's no such a thing as an absolute intrinsic quality, independent of other dimensions such as availability, durability, cost, suitability for a specific purpose, etc.
Mostly, it's extremely naïve to think about "curators" as infallible sources of truth. Curators can be incompetent, and they can be bought -- usually for very cheap.
Eventually, the article asks:
> Who curates the curators?
That's an excellent question! And the answer is: Google. Google points people to Wirecutter, or Booking.com (infinitely more successful than Expedia or Yelp, and not even mentioned).
Google curates the curators.
I think it's very human, when presented with an array of options, to desire the best. Just imagine you are presented with N options for a vacuum cleaner. Would you really just pick randomly? 10 years ago (or maybe more like 20 years ago), there was not nearly as much choice. You couldn't find hundreds of different iterations on the same item. Now that you can, many people spend a lot of time agonizing over the options. I agree that points 2 and 3 don't logically follow from desiring the best though.
I was in camp 'google search is dead' until I was actually shopping for something and then light dawned
google is an ecommerce search engine; all their product decisions make sense if you think of them as amazon without the warehouse, shopify without the checkout
(unfortunately it also puts them in 'more ads, lower margins' spiral that degrades to infinity times infinitesimal)
Breeze search -> https://breezethat.com * Breeze jobs -> https://breezethat.com/x/job-search-beta * Breeze date filter examples -> https://twitter.com/search?q=from%3Adotdotjames%20(date%20OR...
Another obstacle is that the searches Google does worst on are for heavily promoted topics. What company would give you an unbiased answer to "are NFTs good?"
You also state "With strong opinions on how to organize information, reflected in their choice of filters, vertical search aggregators have distinct advantages that horizontal software can never achieve." I strongly believe google can do this, structured data was a big push to make this easier for search engine, but as machine learning models expand to extract structured data from unstructured text, giants will be better positioned to do this then vertical search engine.
"Spotify doesn’t curate what songs make it to their platform. Instead, it takes the entire universe of music and finds endless ways to discover and search across its library, including a mix of manual curation (via playlists curated by its in-house team of curators and its users) and algorithms (like Discover Weekly)." So would the metrics like whether a user follows more links, after the first one, trys another query, hops back to the search results quickly, a form of curation?
And I also think most of your boutique search examples, aren't search engines, but in app searches. A search engine needs to search an index external documents to be a search engine.
Looking at the query, "What do NFT collectors think about NFTs?", how you describe curated search engine would be terrible at answering that, because it would give very limited scope of the curators opinions. Google should return all the results containing content where people talk about this, and you can form an objective opinion based on what people are talking about. A better model for answer this type of question would probably be a massive language library like GPT3, continually trained on current information.
If I’m searching for songs on a music service, I may be interested in sorting by band, by album, chronologically by release, alphabetically by song name, by band member name, etc. (and that is usually possible). Likewise with video, film, books, etc., I want similar capabilities, but am not usually able to find them. So many search engines just leave out this ability. It’s quite frustrating.
If you are working on niche search, I’d love to hear how you are being creative about aggregating and cleaning data to index on. It’s a fun, challenging problem and there are going to be a lot of winners on the other side of this!
This might even help Google know which sites people exclude most and down rank them.
https://news.ycombinator.com/item?id=30347719
https://news.ycombinator.com/item?id=29772136
https://news.ycombinator.com/item?id=29392702
My own anecdata is that non-tech people are beginning to really dislike Google's results. Take a very normal-person kind of search: looking for almost any mainstream recipe. The vast majority of results ramble on for pages about the author's childhood experience and Italian grandmother and how the dog liked the recipe so much, painstakingly repeating the SEO phrases in the currently-most-effective combination. Most people despise this stuff.
I think we all know why that is; SEO is unavoidable, gaming the system is unavoidable. But the important question would seem to be: is Google actually doing a bad job, such that someone could disrupt their model?
I can't see that the answer is anything but "yes". Google is deeply addicted to search ad revenue. That ad revenue comes from the exact same people gaming SEO and producing reams of poor quality content. Google is dependent on its abusers; they are more valuable than its users. How can that end in anything other than a general degradation in quality? And as a public company, Google can't exactly back off from its ad revenue focus for a while and fix its business.
At the same time, nothing I've seen yet could disrupt Google. DuckDuckGo et al are largely just copies, and their differences are too minor to get users to switch.
But it's very odd to say that because disruption hasn't happened, it cannot or will not. That seems like a failure of imagination.
Nobody is saying that. People are upset with Google. They're just recognizing that the arguments made in this article don't make sense. That nonsensicality is heightened by this being, in essence, a venture capitalist's PR statement.
Also, the blog post is republished from somewhere else, not written by a16z.
A combo of curation and filtered google/bing results.
For curation, pay people who are into whatever niche topic and have a review process that spots and eliminates collusion and corruption.
And where you don't provide curated links give me google/bing results with the crap sites filtered out. Also, upsell me on research librarian services that then feed the curation lists.
links:
although run Breeze search, I sometimes explicitly will search on Bing / Google to see what ads pop up when shopping for something haven't ever bought before as part of product discovery
Breeze search -> https://breezethat.com
* Breeze jobs -> https://breezethat.com/x/job-search-beta
* Breeze date filter examples -> https://twitter.com/search?q=from%3Adotdotjames%20(date%20OR...
* @dotdotjames on twitter for DMs / etc
We had thousands of users stopping through over the months but that's not enough for an AD supported model and ultimately indexing the web with a significant enough amount of coverage so that results weren't lacking and then working on a novel enough ranking methodology that combined vector search signals with classical search signals is a massive undertaking that was running us thousands of dollars a month.
It's possible but you need to basically be prepared to burn a lot of money, more than I and my friend were willing to support. That's why most of these alternative search engines serve custom Google results combined with other aggregator content or license Bing and they produce good results but at the end of the day your still beholden to the index and to some degree the ranking signals of the engines your building over.
So far in the landscape Kagi is doing a good job on establishing itself with it's use of lens and subscriber based support for it's boutique engine. Neeva and You.com are also both in that space and have the benefit of being supported by gobs of funding (Neeva in particular due to the background of the founding team.)
All of these engines are attempting to provide the curated experience this article talks about in the form of programmatic boutique like searches over Google and Bing's existing index. From a technical standpoint this makes obvious sense.
But.
From the year+ of working on that search engine and my continued interest in alternative search engines here's the thing. Google and Bing index web pages, but there are so many alternative forms of curated information we've created over the last decade and a lot of that information is in semi-public spaces like slack, discord, whatsapp, telegram, facebook groups, newsletters etc.
As more and more spam has propagated over the internet real useful information and discourse has retreated into these semi guarded spaces that the existing incumbents do not index and do not serve which means all of these respective engines are missing that information which means in turn you're missing that information regardless of where you search.
Want to buy a car? The best review of what car to buy is probably happening in a car enthusiast discord or slack maybe an e-mail newsletter. But you won't see that. So my opinion? I think the next real killer search application is going to figure out how to index and make that content searchable not an alternative take on searching web pages.
That's called Deep Web("semi-public spaces like slack, discord, whatsapp, telegram, facebook groups, newsletters etc.") and you can't crawl and index it unless you partner with them and they allow you to do it.
But the biggest question that keeps haunting me is Does the world need another internet search engine? Casual users just don't care if your new internet search engine is 10 or 20% better than Google, what I'm trying to say it needs to be something totally different and significantly better in order to catch the attention and the engagement of the users.
It also makes more sense if users aren't the product and you can have a business doing 10's-100's of millions in revenue based on serving some niche without having the be exploitative via subscription or some b2b offering you make. Google is so big it's almost a certainly they are more than suboptimal at a profitable enough niche. DuckDuckGo proved that with it's focus on privacy which is most certainly a niche and the 100million in revenue they do.
Personally I think there's a lot of B2B opportunity for a lot of this information where the search can be a multi billion dollar business just not on the backs of consumers.
I thought about this and I disagree. Indexing discord/slack (even if would be possible) would probably just add a lot more noise. Yes there might be an occasional gem in a discord channel discussion, but vast majority of time it would be low quality content/chatter. There is another problem - how do you rank it? If someone says on Discord that earth is flat how do you know they are right or wrong? What signal can you use - emoji like count? How do you rank this result vs a wikipedia result?
It is still more likely that the best review of a car will come from a specialized car review site (that is not ad-ridden).
You'd search it for opinion content and I think for that it'd be up to you to discern the quality the same way it works when you stumble on a reddit post with relevant content but since it's a niche question you don't have a flood of up votes to legitimize it which is how a lot of my reddit like searches go. I've found perfectly valid answers to questions I have on reddit posts that have no or few < 5 upvotes, something like 60% of the time I wind up on a reddit post. So to bring it back to discord maybe positive emojis are a signal in that case.
In the case of cars to continue my example, If I see a discord message in a search that is like "I own a 2021 Toyota Corolla Hybrid and I have x y z concerns with it." it's an opinion you can choose to value based on ownership.
But I think asking a search engine, even a search engine like google, to make a value judgement on an opinion is asking a lot and not something they really should do. A user has to have some culpability in the information they consume.
It'd be pretty trivial to negatively downrank though after a user interacts with a result and finds it lacking.
There are proxies for quality of opinion but I think even now with google it's largely up to us as users to discern that especially on places like reddit which are ranked highly by google because it's reddit but don't qualify anything beyond that except in all but the most crowded of subreddits.
I think general fact based search is super solved and competing there is a waste.
Edit: And when I say general fact based I mean like "Who is the president of the US right now?"
For individual areas of knowledge (programming being an obvious one for me) there's still improvement to be had for sure.
This is where a savvy entrepreneur would thrown in "AI-augmented search technology" to really get VCs salivating.
I have a lot of thoughts on how to capitalize on that and especially after the latest HiQ ruling and consulting with a lawyer who specializes in this type of thing on how to go about indexing that type of information.
Happy to talk more over e-mail if that piques your interest. One of the e-mails I check is in my profile.
IMO Google got lucky that they were able to profit from this kind of valuable information being out in the open in the early days, before content producers wised up to it. I think the gates are increasingly closing shut, and TBH I hope it stays that way. No one should have the right to just take this kind of value without giving back equally. Ostensibly Google has been giving back by being an excellent search engine but this seems to be on the decline.