Hacker News Reader

No, actually, I as an adult human being, am quite capable of using search, to find what I want.

Now, I agree, search is also governed by algorithms - but I'm also quite capable of sitting there for a few minutes, paging through until I get past them to find what I want. I do not want anything controlling my filters, except me. Edge cases can be dealt with. Latex lovers need to learn that it's a text editor first.

As for Amazon, I have never found their recommendations helpful, or bought anything via them. If Amazon wants to help me, it really needs to clean its house on fake recommendations, and bogus products.

A simple google search for, say, "string" confirms to me that what you say is not true. I program, so it shows me ... well not clothing. Things I would not want to sort through.

So you should realize that even something as simple as search cannot work well without "using my information". And I very much want it to do that.

If that were true, people would not be able to find anything useful on DuckDuckGo.com and similar sites. Tracking is definitely not a prerequisite for quality search.

https://duckduckgo.com/?q=string&t=h_&ia=web

https://www.google.com/search?q=string

Judge for yourself whether context matters. I suppose the results will differ from person to person, but I feel comfortable saying that while Duckduckgo isn't a lost cause, it contains a lot of bullshit to filter through. Example:

> STRING: functional protein association networks > STRING is part of the ELIXIR infrastructure: it is one of > ELIXIR's Core Data Resources. Learn more > ...

I mean ... really ? I suppose it must be a string, but, and maybe this is just me, I'm not looking for genetic manipulation tips or databases when I search for "string".

What I find funny, is that you think some crappy algorithm, can beat the real neural network, your mind ;)

You convert a fixed problem ('locate search terms that have a high correlation to what I am searching for') to a fuzzy problem of "how can I guess what this person is searching for, based on a sequence of search terms, where I manipulate the results, thereby gauranteeing that the search sequence is poisoned by what I presume they are searching for').

Oddly enough, the latter, is how you get high dwell on a search engine (COUGH COUGH).

as soon as I started using duckduckgo - bang, back to 1 minute searches as I just altered my search terms and actually got back, what I wanted.

Is it called "dwell"? Because it's a useful word for an incredibly annoying trend that Google were doing the last few months I used it: whenever I searched for a word I'd used in a previous search they would try really hard to get me the same results as before, including ignoring quoted words and minus-prefixed strings. At that point I just really wanted Google to forget my previous searches, but of course that's impossible. Since I'm interested in more than one thing, using a different search engine is just better.

On a related note, Google seems to be doing something similar with startpage.com searches, I guess because mostly geeks use it. Or else they are somehow getting enough data through startpage.com to identify me.

dwell is a technical term from production lines ;) it means you adjust the speed at which your conveyer runs, to offer more time under a particular process (implemented along the line). typically heating.

Google, like Amazon, and Apple, have jumped the shark.

Facebook, is the shark. ;)

Actually, when you link up 2 neural networks, or even a neural network and, say, a search algorithm, the neural network rapidly learns to use the external algorithm to do what it was planning to do anyway.

So there is no "beating" a neural network. The reward for having such a search algorithm is simply the ability to put thoughts into your mind. Your mind may still reject them, and if you don't feed it enough relevant data, it will rapidly abandon you.

presuming you don't just abandon the search is irrelevant. :)

which is why I don't stress that "agree to disagree over similar premises". can be both a hindrance, and a benefit.

string c++

The human mind is far greater than the sum of the algorithms - when we allow it to be.

I do remember that from the 90s. There were whole books with google search tips. The extra syntax that you needed to know.

The meaning of quoting search terms (all 3 different ones). The + and - things: +term -term. The special syntaxes "site:", the use of asterisks within quoted search terms, "define:", "link:", "related:", "...", "nearby", "filetype:", ...

And then the many "soft" tips, like leaving out unimportant words and so on.

I mean it was ... euhm ... well I remember it was all kinda cool (and it still works !), but ... today it is better.

Well, I'm not trying to make the case that recommendation algorithms are strictly unhelpful. It would be absurd to imply that there aren't many cases where they have provided a benefit to the end user. I just think that in many cases in my experience they have a bad signal to noise ratio to the point where it annoys me, because of the limitations of surreptitious data collection. Also, Youtube and Amazon don't allow you to turn recommendations off, which to me implies that in a choice between user preference and their business interests, it is clear which one is more important to them.

> That's exactly what these algorithms do. They try to find what YOU would find meaningful and fulfilling and they really do try to give it to you. When that means money will be exchanged they prefer one vendor (that has paid them) over another (which hasn't, or paid less), but does that really change anything ?

To me, this question is asking, "Is there a difference between a data collection algorithm whose primary goal is to help me, and a data collection algorithm whose primary goal is to profit off of me?" While there does exist a subset of transactions where the results would be the same, this whole article upon which we are commenting provides ten case studies of unintended harms or instances where the interests of the data collector and the interests of data collectee are not aligned. If you are looking for a more in-depth discussion illustrating some of the potential harms, I found Robert McChesney's Digital Disconnect to be a convincing first look at the problem, and there are some good documentaries on Thoughtmaybe that may be relevant as well, including Advertising at the Edge of the Apocalypse, Weapons of Mass Surveillance, and Adam Curtis's The Century of the Self.

> To me, this question is asking, "Is there a difference between a data collection algorithm whose primary goal is to help me, and a data collection algorithm whose primary goal is to profit off of me?"

Unfortunately you are describing a choice between A and B. And B does not exist. It would not make a profit, and therefore nothing short of the government providing better search can make it happen, paid for by taxes (Google's operating budget is ~$80 billion dollars, which would have to come out of your taxes). We both know we don't want that to happen, so ...

Given that you're describing a choice between A (for profit recommendations) and nothing, there is but one choice.

Well, there's 2: data collection algorithms that profit "off of you" or no data collection algorithms at all. No context sensitive search, regular lingerie of the wrong gender on your amazon frontpage, baby diapers on the front page of walmart.com when you've long since retired, ...

Somehow people used libraries just fine without having a personal profile based on regressions governing the card catalog. I really can't empathize with this sense that shopping and watching video channels 'can't be done' without recommendation algos.

You say that as if, economically speaking, it is a realistic option to go back to the past. It isn't.

In theory we could go back to heating with open fires. Steam engines. And so forth and so on. We can't. It wouldn't work, and with steam engines it's blatantly obvious why, and I get that with search engines it's not that obvious, but it's the same reason.

Actually there's some pretty cool stuff going on in steam engine research, if you look up "Coalition for Sustainable Rail", but that's neither here nor there.

I'm still not sure what part of filtering search results explicitly instead of statistically/nondeterministically is so backwards to you. Really, what's your angle in insisting on this particular future, where everything is sorted based on personal profiles?

That may be true, but I think you'll find that's further forward movement, not backward movement. I'm sure that aside from the operating principle, there are going to be huge differences between new and old steam engines.

One thing I know about, for example, is nuclear power generators. They are "steam engines", sort-of. That requires a creative interpretation of how it works, as it doesn't match at all.

> Fact is you cannot use these sites realistically without these algorithms.

Really? I have my web browsers set up to always clear all cookies (no exceptions) when I close them (and I don't keep them open long). I also use various browser plugins to get rid of other means of tracking as far as possible.

And I don't even have a YouTube account to log in to. Still, I use the site often and generally find what I'm looking for.

Similar on Amazon, where I have an account, but I only login when I have put things into the cart and actually want to checkout. Until then, again, Amazon knows nothing about me. But I find the things I look for. Oh, and despite being an Amazon customer for 15 years or so, with plenty of buying-history for them to analyze... their recommendations are practically always uninteresting. So I don't think I'm losing out on anything...

>... their recommendations are practically always uninteresting.

I've had Amazon recommend books that I've previously purchased from them.

> Fact is you cannot use these sites realistically without these algorithms.

I don't rely on recommendation systems or past activity. I search specifically for what I want and find it without difficultly. I don't need to be told what I want. I don't see the value in being babysat or shepherded by an algorithm.

To quote Alexis de Tocqueville from his book Democracy in America (a bit out of context):

"What good does it do me, after all, if an ever-watchful authority keeps an eye out to ensure that my pleasures will be tranquil and races ahead of me to ward off all danger, sparing me the need even to think about such things, if that authority, even as it removes the smallest thorns from my path, is also absolute master of my liberty and my life; if it monopolizes vitality and existence to such a degree that when it languishes, everything around it must also languish; when it sleeps, everything must also sleep; and when it dies, everything must also perish?"

He was talking about the government. But in a world of massive and pervasive corporate surveillance and in a world of advertising more finely tuned to manipulation, I think we can also apply the quote to the likes of Google and Amazon.

> He was talking about the government

Exactly ... and you're proposing to use the government (expanding it's survaillance, btw) to hold back these private corporate surveillances ?

Many of de Tocqueville's objections don't really apply to this situation by the way:

1) google isn't pervasive, nor is facebook or amazon. In reality there's many thousands of search providers. Most are far worse than Google, Amazon, Facebook in the tracking department, and some are more-or-less government departments

2) "when it languishes, everything around must languish, ..." this is a complaint about lack of competition, and this online space isn't lacking for competition at all.

And lastly, I do wonder what you think about the alternative I think you're proposing. Having the government provide and regulate these services. I might argue that that would be far worse in the surveillance department, but we could also just observe that there are 2 countries that actually do this. Russia with Yandex, and China with Baidu. Both are more-or-less government departments. Both governments' security departments feel the need to have people's search terms associated with their names available for the secret service and the police is a necessity. It is not hard to find online reports of western security services asking for and getting access to entire browsing histories [1], so realistically under a government search engine they'd get search term surveillance. Do you think people using these search engines that just report everything to the government are better off, surveillance-wise ... or worse ? Because that's what would happen under your proposed alternative.

It's sad, but realistically the only alternative available under the current governments, in the US, or in Europe, or anywhere else for that matter with one or two exceptions, is government surveillance instead of splintered private surveillance. I think I'll take the private option, thanks.

[1] It's browsing history, not search history, but ...

https://www.theguardian.com/uk-news/2015/oct/30/police-seek-...

https://www.independent.co.uk/life-style/gadgets-and-tech/ne...

https://www.theguardian.com/commentisfree/2013/aug/01/govern...

and you're proposing to use the government (expanding it's survaillance, btw) to hold back these private corporate surveillances ?

No. Re-read what I wrote.

So what exactly is your enforcement mechanism for stopping this surveillance ?

I have my browser setup to block the recommended videos column on Youtube and it's a lot more enjoyable/useful to me. I'm not really sure what you're trying to say.

> You realize that you're just complaining "they" are not using your data well enough. You certainly can't realistically sort through youtube yourself, nor can you realistically find what you need on Amazon.

This implies that I need to be nannied by Google or Amazon in order to use their websites.

Somehow, I doubt that.

> Fact is you cannot use these sites realistically without these algorithms. It just can't be done.

There was a point, not very long ago, where these websites had no, or naive, recommendation systems. Search functionality was based on strict to fuzzy keyword matching.

Both YouTube and Amazon were infinitely less frustrating to use back then.

That’s because back then their results weren’t optimized for engagement.

> You realize that you're just complaining "they" are not using your data well enough. You certainly can't realistically sort through youtube yourself, nor can you realistically find what you need on Amazon.

> Fact is you cannot use these sites realistically without these algorithms. It just can't be done. Or at least one can say, you can use Amazon and Youtube better when guided by these algorithms. I do think that it is the difference between not being able to use them at all and using them, but maybe you think it's just a bit less than that.

> (also logical positivism and inferring things through statistics are pretty much diametrically opposite things. No logical positivist is ever going to accept statistical evidence about anything, but no matter)

> > I suppose I can imagine that if we lived in a society free from exploitation, and someone has generated an algorithm that could help you find meaning and success and fulfillment based on all the tracking data.

> That's exactly what these algorithms do. They try to find what YOU would find meaningful and fulfilling and they really do try to give it to you. When that means money will be exchanged they prefer one vendor (that has paid them) over another (which hasn't, or paid less), but does that really change anything ?

-amazon is only unusable without their algorithms because their search and the product data (indexes, categories, refinable factors) backing it is utter crap. Instead of building Alexa -amazon should have made their search adequate for the previous decades standards.

Or at least one can say, you can use Amazon and Youtube better when guided by these algorithms

In the case of Amazon when I search for "headphones" there's plenty of data in my history on there that could aid a product recommendation - they know what music I like and what devices I use to listen to it via Amazon Music, they know what mobile phone I have from the Amazon app, they know that I like the Anker brand from what I've bought recently, they know approximately how much I spend on gadgets, they know I only buy things that I can get Prime delivery on and they even know which headphones I bought last time. And yet the search listing doesn't have the things I'd like most at the top. Or even in the top ten.

If they're trying to make the search better for me then they have failed completely.

"...you cannot use these sites realistically without these algorithms..."

We're past peak recommenders. There's very little juice left to squeeze from these turnips.

Emphasis is now on better search. Improve the tagging, use some ML to better categorize stuff, leverage personalization to boost & filter results, and misc secret sauce.

Source: Work on recommenders for a retailer.

> Fact is you cannot use these sites realistically without these algorithms. It just can't be done.

You never used your browser's incognito mode?

>You certainly can't realistically sort through youtube yourself, nor can you realistically find what you need on Amazon.

People will find what they need, period. The algorithms are tuned to show you the items you’re most likely to buy/watch on top of what you actually came for.