evolvingstuff

That's interesting. My brain does the same in terms of attempting to simulate that model of the people, but when it becomes impossible to run the algorithm (too large of a group), instead of just giving up, my brain goes into panic mode. I have found propranolol helps me with many of the physical symptoms that would otherwise dominate.

Very excellent point. One simple possibility I could imagine to mitigate this would be having an additional device/button one had to physically press in order for the thoughts to be vocalized. I think that would be an extremely simple but clear indication of intentionality.

From the study:

"It should be remarked that, in the majority of the cases, people adopting plant-based diets are more prone to engage in healthy lifestyles that include regular physical activity, reduction/avoidance of sugar-sweetened beverages, alcohol and tobacco, that, in association with previously mentioned modification of diet [62], lead to the reduction of the risk of ischemic heart disease and related mortality, and, to a lesser extent, of other CVDs."

"It has also been described that vegetarians, in addition to reduced meat intake, ate less refined grains, added fats, sweets, snacks foods, and caloric beverages than did nonvegetarians and had increased consumption of a wide variety of plant foods [65]. "

You are correct, that is an error in an otherwise great video. The k+1 token is not merely a function of the kth vector, but rather all prior vectors (combined using attention). There is nothing "special" about the kth vector.

I suspect in the near future there will be a number of cases where individuals will intentionally release a bunch of AI "chaff", in the sense that having a very large number of bad videos/texts about them, of which many are clearly false, will disguise the actual bad behavior. I'm not sure what term/phrase will be used for this particular tactic, but I am absolutely certain one will arise.

I love Nomnoml. I have lately been using it to visualize hierarchical tag structures in a browser-based PKM project I am working on. I find it creates fairly clean layouts.

Example:

https://imgbox.com/9A1mDyNv

Why would RWKV have a particular advantage in this context? (I may be missing some key intuitions)

You have been shamelessly self-promoting your Hopf algebra/deep learning research on a very large percentage of posts I have seen on HN lately, to the degree that I actually felt the need to log in so as to be able to comment on it. Please. Stop.

Easy. Have a setting where someone can consent to (or not) receiving images that may potentially have nudity in them.

I have found the books System Design Interview (volumes 1 and 2), to be really nice and informative reading material on this topic.

Maybe I'm missing something obvious, but this seems like a way to calculate a mode, not a median.

So I just googled "differentiable Monte Carlo" and it looks like that concept has so far only been applied to ray tracing, but I would be shocked if that (or similar) doesn't become a thing in the next year or three, such that AlphaGo et al become end-to-end differentiable.

Thank you!

Completely agree! Plus, less trivially, there can be a bunch of different link weight settings (for an assumed distribution of inputs) that result in nearly-symmetric behaviors, and then that is multiplied by the permutation results you have just mentioned! So, it's complicated...

Here are some "animated" loss landscapes I made quite a long time ago:

http://evolvingstuff.blogspot.com/2011/02/animated-fractal-f...

These are related to recurrent neural networks evolved to maximize fitness whilst wandering through a randomly generated maze and picking up food pellets (the advantage being to remember not to revisit where you have already been.)

May have looked it up.

Back in the early 2000's (when I first started getting into this stuff), Schmidhuber's ideas were absolutely off-the-charts revolutionary compared to anything any other lab was doing. The level of creativity coming out of IDSIA was insane. I truly believe he has not been given even slightly the level of recognition he deserves. Many of the original founders of DeepMind are his direct students. E.g. Alex Graves, who is basically the pioneer of "attention" in neural networks (and we all know what architecture that led to...)

If I remember correctly, a big advantage of particle filters over Kalman filters is that the particle filters can model complex, multi-modal distributions, whereas the Kalman filter just updates the mean and covariance of a single high-dimensional Gaussian (albeit optimally). This is nice for localization problems because it allows you to keep track of multiple competing hypotheses about your current location.

I am acquainted with a person working on this: http://code.google.com/p/openworm/

Another fun fact about high-dimensional spaces: Randomly pick k points in an n-dimensional space. Now, find their average location. As the number of dimensions increases, it becomes progressively more likely that every point will be closer to the average than it is to any other point.

Agreed. To elaborate:

Saccades to an unexpected stimulus normally take about 200 milliseconds (ms) to initiate, and then last from about 20–200 ms, depending on their amplitude (20–30 ms is typical in language reading).

Saccades of 20ms in duration are ones that are very near to the current center of focus (e.g. moving to the next chunk of letters while reading the words of this sentence). This just means that detailed rendering needs to extend to a slightly larger radius, but this is still significantly cheaper to render than an field of view. For larger jumps there is ~200ms during which the computer can attempt to predict the final destination of the saccade, and thus begin to do some preemptive computations. Once the saccade lands at the new location, assuming a rendering speed of 100fps, there would be at most 10ms before the high-res version kicked in, but again, with some degree of preemptive/predictive computation, perhaps a slightly better version could be available immediately.

Only twice as unreasonable :)

One way the circumvent the need for 5000 teraflops mentioned in the article is to exploit the fact that our eye is only capable of seeing a relatively small area in great detail at any given instant. Per viewer, we only need to finely render a tiny bit of the scene. This of course precludes the same level of realism on a shared screen, but I suspect that viewer-specific devices (e.g. http://www.scientificamerican.com/article.cfm?id=virtual-rea... ) will become the norm as we move further towards virtual reality.

Anyone else not really care for the line break in the title? Seems to me like it would have been better after the colon, so that "Minimalist Web Design" would be grouped together on the next line.

I wouldn't normally be so nit-picky, but it is an article about typography after all.

Standard hill-climbing would indeed work better under the assumption that the fitness landscape is fairly unimodal, in terms of local optima. In this problem that may or may not be the case. I think that this is the motivation for a somewhat more "global" search strategy (something that can escape local optima) like genetic algorithms.

That being said, amongst the possible candidates for search strategies, genetic algorithms are fairly lousy. Differential Evolution or CMA-ES would likely work far better.

And another: http://www.earslap.com/projectslab/otomata?q=0q0o88801y1f7b7...

This is quite fun to play around with! Many of the other CA-based music generators I've seen have sounded rather atonal and random compared to this one. Here's one I made using symmetric initial conditions (doing this makes for somewhat interesting visual patterns as well): http://www.earslap.com/projectslab/otomata?q=0q1q2q3q8w8y8p8...

There may indeed be more distracted/stupid people, but the inebriated ones cause about twice the number of deaths overall in the US:

http://www.cdc.gov/MotorVehicleSafety/distracted_driving/ind...

http://www.cdc.gov/MotorVehicleSafety/Impaired_Driving/impai...

Very interesting video, these concepts seemed to have progressed a lot since last I checked in on them.

That being said, I'm finding it very difficult to find any objective comparisons of these algorithms to other, more mainstream machine learning techniques. In the talk, he gave the impression that there was a tremendous amount of data to back up these claims, and that he just didn't have time to present it all. I went through many of the white papers available on the Numenta website. Many were just overall outlines of the approach. A few of them demonstrate tasks for which some form of learning is occurring, however, it was hard for me to know, in the absence of objective comparisons to other techniques, just how good the results really are.

So far, the only objective comparison I could find involved handwritten character recognition, and that was against what appeared to be a standard feed forward neural network with only a single hidden layer. Not exactly state of the art.. why not compare to SVMs, convolutional NNs, deep belief nets, etc..?

So I am at this point hopeful, but fairly skeptical. If nothing else these are some inspiring ideas.

Thanks fubari, great information!