219 karma · joined September 9, 2021
a good example is discovery of attention mechanisms/transformers replacing more cumbersome and computationally expensive RNNs and LSTMs in NLP and more recently outperforming more expensive models in computer vision.
It's a continuation of Plate [6] and Kanerva work in the 90s and Olshausen' groundbreaking work on sparse coding [7] which inspired the popular autoencoders [8].
I find it especially promising they found this superposition based approach to be competitive with optimization so prevalent in modern neural nets. May be backprop will die one day and be replaced with something more energy efficient along these lines.
[1] https://redwood.berkeley.edu/wp-content/uploads/2020/11/frad...
[2] https://redwood.berkeley.edu/wp-content/uploads/2020/11/kent...
[3] https://arxiv.org/abs/2009.06734
[4] https://github.com/spencerkent/resonator-networks
[5] https://en.wikipedia.org/wiki/Sparse_distributed_memory
[6] https://www.amazon.com/Holographic-Reduced-Representation-Di...
[7] http://www.scholarpedia.org/article/Sparse_coding
[8] https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf
The problem with this sort of "fact checking" is that its keepers are biased as well. So you need a "fact check" on a "fact check" which might lead to stack overflow eventually. Unless you can objectively judge the accuracy yourself. And track record means nothing if say a reputable news outlet is purchased by a billionaire with a political agenda.
I think many people break through the walls of their echo chambers when they stumble upon some basic inconsistency and discover they are being lied to by seeing e.g. raw footage of an event vs. its media interpretation.
For many people however finding the truth is not interesting or important enough. I guess echo chambers are for those who deserve them.
Chauvin's lawyer did a decent job, but it was a travesty of justice with politicians demanding "the right verdict" or else cities will burn and aggressive mobs outside intimidating the jury, not to mention an actual BLM activist on the jury. It was a 'banana republic' style trial.
The legal system is as political, biased and arbitrary as any other part of our lives. There is no justice, only luck.
I'd explore all these, may be do a few months stay in each city, that would be really cool experience you could tell stories about.
Btw if you go further east check out Chernivtsi, it’s a hidden gem in the East Carpathian mountains.
2-bd start at around 500 EUR/month utilities included, but can go higher if you choose a good location and bigger apartment. For 600 EUR you should be able to find a solid one, with luxurious ones in good location starting at 750 EUR/month.
Good internet speed, friendly people, european culture, quick flight or train ride to anywhere in Europe, pretty girls.
https://www.reddit.com/r/scihub/comments/ovp5c0/make_scihub_...
edit: here it is https://news.ycombinator.com/item?id=8022414
Facebook and other social media are definitely partly responsible for the outrage culture and divisiveness plaguing the country.
But I assume it’s a difficult problem for algorithms to detect political agenda, as it seems to pervade every aspect of news these days.
Eliasmith created "Semantic Pointer Architecture" which is sort of a crossover between CS and CogSci https://en.wikipedia.org/wiki/Spaun_(Semantic_Pointer_Archit..., also see his book on "How to build a Brain"
I wish the current AI research would get unstuck from local minima of deep learning, there's been so much progress in cognitive sciences in the last few decades.