researcher11

I didn't downvote so I don't know why people are doing it directly but I just checked and Squarespace has a very low glassdoor rating. Similar to that of Theranos before the scandals. There might be some hints there.

Sure, I wear blue blockers 2 hours before bed (~11pm) and I use blue LEDs that shine in my eyes from a distance while I'm reading for an hour after I get up at (7am). I don't use an alarm clock and I can't drink caffeine (Graves). I found the blue blockers at night to be more effective for regulating sleep than the blue light in the morning. You can get blue blockers for $5 on Amazon. Highly recommended.

I also have a very strict microbiota tailored diet and do intermittent fasting (5:2). I do this to naturally manage Graves' disease. I also have to exercise regularly and I use blue light therapy to regulate circadian rythims. It's not easy but it really works.

The alternative was to remove the thyroid and replace the hormones with synthetic. Unfortunately you body needs different amounts for different things at different times of the day. It is a constantly moving target and in many cases you never really feel right again.

The special diets were known to the Graves community when I started 8 years ago. Later on I learned and incorporated microbiota theory and used that as a basis for experimenting which has worked well. I only recently started intermittent fasting and feel even better for it.

Interestingly I started intermittent fasting after reading Nassim Talebs book Anti-fragile. He and I share the same opinion on the medical establishment (I won't bore you with my details) so I figured I'd give it a go and it helps.

This is my favorite Bad TED talk - by a physicist using the power law straight line trick; https://www.ted.com/talks/sean_gourley_on_the_mathematics_of...

It was also published in Science. Sean also founded Quid.

I thought I was taking crazy pills when I saw it. I wondered how could this guy not know better and how could no one else around him know either.

The people that I know at MS and Yahoo told me that Page Rank was never as good as meta-documents. And meta-documents was more than sufficient to explain the improvement in relevance.

Of course this is anecdotal and I would be happy to see evidence to the contrary.

It's basically the union of Information Retreival, Big Data, and Machine Learning. There is a lot of good and bad info out there.

It's best to tailor your learning around a problem so you can get feedback on what works. Building bespoke search engines is expensive and often not worth it until the problem is big enough. Machine Learned optimization is even more expensive.

So in the likely instance that your problem is small; I'd stick to an off the shelf Lucene. If you need more specialization write plugins for it. If you need more speed then DIY OkapiBM25 in native (maybe Rust these days). If you need Big Data I'd use Spark. If you need ML then GDBT in R. If you need advanced NLP then Deep Learning. At each stage it's usually diminishing returns. So quit once you start losing money on the additional effort.

Edit: As a fun aside. Page Rank doesn't work very well and AFAIK Googles major advance was from creating 'meta documents' using anchor text and search queries. Google has a habit of sending out red herrings to guard their important ideas. So if some blogger is waxing lyrical about Page Rank you know they're full of it.

I'm an applied researcher in search science. This is a very typical domain specialized search problem. These days it uses Big Data, Machine Learning, constant experimenting / prototyping, A/B testing. I get to geek out each and every day. It is my dream job :)

I consider it one of the main reasons Apple lost so much content market share to Amazon despite Apples first mover advantage - Amazon has better search and you can actually find the stuff you want. AFAIK Apple tried to start up an applied research group to fix the problem but they refused to pay enough and ended up with the dregs. They tried to hire me 3 years ago for $130K. It highlighted that they didn't understand the importance of the problem and we're likely to fail. I typically make ~$240 as FTE and more as a contractor. Apple also has a culture of secrecy where as search science is pretty open.

Anyway; I checked out Steam. The search there is clearly very poor and can easily be improved. The overreliance on tags is usually a good indication that they are not digging into their usage data for their recommendations.