stanfordkid

It's better than using randomly initialized weights. It's more of a theoretical exercise to explore biology. When an infant is born maybe the visual cortex already has some notion of edge detectors etc. through a system such as this one despite never having really opened it's eyes.

I did a similar project but using 3D fractals I found on shadertoy feeding into ViTs. They are extremely simple iterative functions that produce a ton of scene like complexity.

I have a pet theory that the visual cortex when developing is linked to some kind of mechanism such as this. You just need proteins that create some sort of resonating signal that feed into the neurons as they grow (obviously this is hand-wavy) but similar feedback loops guide nervous system growth in Zebra fish for example.

It's pretty simple... the word circle and what you can correlate to it via english language description has somewhat less to do with reality than a physical 3D model of a circle and what it would do in an environment. You can't just add more linguistic description via training data to change that. It doesn't really matter that you can keep back propagating because what you are back propagating over is fundamentally and qualitatively less rich.

I'm curious to understand, why would you build your website this way vs. say jQuery. I've never really understood the HTMX ecosystem. Is this just to avoid javascript and replace that with html pages, id's and attributes? It feels like the DOM is a very clear abstraction and scripting is a more powerful way to manipulate it. What do people like or prefer about this approach and paradigm?

I'm not sure I totally buy the "no plasticity" argument. If you are allowed to write to context in an agentic fashion, certainly the LLM can record intermediate answers, go back and re-rank it's memory. The "plasticity" is in the form of data that can be looked up and referenced as a shortcut. I would think this forms a Turing complete system so theoretically it can represent pretty much anything.

Isn’t this kind of all bullshit. Like Anthropic licenses so many of its models through Bedrock. If the DoD has a contract with Amazon they can just use them.

Shouldn't public signature of the hash of the exe file from a known key before execution fix this??? What am I missing?

Just look at the domains. Obviously social media will get harder to do this with, maybe that's okay though. I think a simple criterion can be used: could the pre-trained LLM have come up with this itself? If so it probably doesn't have training value.

Cool to see some innovation from HP, this is actually unique and fills a niche of going from desk to desk (home->office), without needing mobile. Much better than ripping off the latest aluminum MacBook designs.

I don't see how you get around LLMs scraping data without also stopping humans from retrieving valid data.

If you are NYTimes and publish poisoned data to scrapers, the only thing the scraper needs is one valid human subscription where they run a VM + automated Chrome, OCR and tokenize the valid data then compare that to the scraped results. It's pretty much trivial to do. At Anthropic/Google/OpenAI scale they can easily buy VMs in data centers spread all over the world with IP shuffling. There is no way to tell who is accessing the data.

Is it really true that nothing would change if the Sun's mass was suddenly compacted by several orders of magnitude (into a point mass or black hole)?

This seems unintuitive to me. The sun is a million miles in diameter, so surely shrinking that to zero would lower the amount of gravitational force infinitesimally since the gravity is 1/distance^2 not linear. I would think the planets would sort of drift ever so slightly farther.

The dude works for GitHub. I don’t doubt there is some rotten code on there, but what you’re saying seems like a stretch and exactly what he’s describing.

To be fair, millions could be hundreds of millions.

Just because it uses the className attribute doesn't really mean it is "like tailwind"... SQL is not anything like CSS classes and cannot be composed in the same manner. It's basically just using className as a data attribute. You might as well just stick raw SQL in there and parse it... what is the point of the weird hyphenated pseudo dialect?

What’s the point of this article — everyone knows desiring heroin is different from wanting to become an Olympic swimmer.

This feels like a nascent form of what Nick Land describes as "hyperstition" ... feedback loops where the presence of it's idea within peoples mind brings it into realization.

Worth a read: https://en.wikipedia.org/wiki/Hyperstition

I mean... in what world would you send a customers private root key to a web browsing client. Like even if the user was authenticated why would they need this? This sort of secret shouldn't even be in an environment variable or database but stored with encryption at rest. There could easily have been a proxy service between client and box if the purpose is to search or download files. It's very bad, even for a prototype... this researcher deserves a bounty!

They didn't even mention MessagePack. Also there is a huge amount of developer over-head for using things like ProtoBuf. You can always validate your API responses with Zod or JSONSchema so that is a bit of a moot point!

I’d argue that AI apis are nearly trivial to switch… the prompts can largely stay the same, and function calling pretty similar

I agree with his take on Googles enormous strategic advantages.

I think he’s wrong that OpenAI can win this by upping the revenue engine through ads or through building a consumer behavior moat.

At the end of the day these are chat bots. Nobody really cares about the url and the interface is simple. Google won search by having deeply superior search algorithms and capitalizing on user traffic data to improve and refine those algorithms. It didn’t win because of AdWords … it just got rich that way.

The AI market is an undifferentiated oligopoly (IMO) and the only way to win is by having better algos trained on more data that give better results. Google can win here. It is already winning on video and image generation.

I actually think OpenAI is (wrongly) following Ben’s exact advice — going to the edge and consumer interface through things like the acquisition of things like Jony Ives device company. This is a failing move and an area where Google can also easily win with Android. I agree with Ben that upping the revenue makes sense but they can’t do it at the cost of user experience. Too much at stake.

What do you mean by “understand”? Do you mean conscious?

Understand just means “parse language” and is highly subjective. If I talk to someone African in Chinese they do not understand me but they are still conscious.

If I talk to an LLM in Chinese it will understand me but that doesn’t mean it is conscious.

If I talk about physics to a kindergartner they will not understand but that doesn’t mean they don’t understand anything.

Do you see where I am going?

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal models entirely. He also pretty much discounts everything that’s happened with AlphaFold, Alpha Go etc. reinforcement learning etc.

The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. Is the brain NP complete? Maybe maybe not. I could see many arguments about why modern research will fail to create AGI but just hand waving “reality is NP-hard” is not enough.

The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. That’s a discovery on the scale of the internet or google search and shouldn’t be discounted… and usage proves it. In 2 years there is a platform with billions of users. On top of that huge fields of new research are making leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc.

It’s a paradigm shift.

IMO This is a dangerous way of thinking. This is the Timothy Leary adage that you are stating. It is not just a "set and setting" issue. Whether or not you have a good or bad trip, or whether you are alongside kind empathetic people or whether what you see is positive or negative does not matter. It has a lot to do with the person individual makeup, sensitivity, ego-centricity, and pre-ingrained beliefs. They do not automatically go away after the trip and the realization. The realizations come into conflict with them. This can very easily be a catastrophic event for some.

I personally think Ayahuasca gives an amazing capacity to see the potential within the universe and within ourselves... it is nearly infinite. However if you deeply understand the world and how mis-aligned it and yourself are from that potential it can create tragedy. If you believe you are responsible for this tragedy it can create catastrophe. I don't mean to over-extend my own experiences too far.

What if there is a book that when you read it you would know for 100% certainty when you die there is no hell, and only infinite peace, unless you wanted to go back into hell. Should everybody read that book?

That's what taking Ayahuasca felt like for me. I don't think it is great to do for everyone. I'm curious to hear your thoughts as you have much more experience than me. I had a lot of challenges after taking it.

Absolutely ground breaking and mind shattering book!

I have done Ayahuasca. I think it's dangerous to say it is "healing"... it's a window not a door. You have to be careful, some personalities will develop psychosis and disconnection from reality. You will certainly end up above where you started but there may be a deep trench you have to walk through. Especially amongst type-A highly IQ and rationality driven people this is very likely IMO.

Exactly this. Like is the research actually useful and correct is what matters. Also if it is accurate, instead of schadenfreude shouldn't that elicit extreme applause? It's feeling a bit like a click-bait rage-fantasy fueled by Pangram, capitalizing on this idea that AI promotes plagiarism / replaces jobs and now the creators of AI are oh-too human... and somehow this AI-detection product is above it all.

I agree with you there's a lot hand-wavy jumping to conclusions and claims human effects often feel like pop pseudo-science. However I do think when folks imply that epigenetic effects are negative they tend to point towards things like anxiety and fear. Anxiety very well could have a protective evolutionary benefit if you assume the next generation will also be born in a high-risk environment, but a generation later these qualities would be perceived as a negative consequence.

Another example is famine, it may be advantegous to store visceral fat in a low food environment, but if the famine ends, the next generation is more likely to carry this "advantage" which is now negative: https://www.sciencedirect.com/science/article/pii/S216183132...

Of course I am not saying they are all Nazis. I fully support Ukraine. But certainly many of the battalions have far-right views, as in they are literally neo-Nazi's that support the holocaust / Hitler. I don't mean to use it in a hand-wavy sense. See: https://en.wikipedia.org/wiki/Azov_Brigade

I'm just saying there is a pretty big hypocricy. Calling Hamas anti-semitic etc. while sending these drones to fascist neo-Nazi groups in Ukraine. It is a tiny tiny group, but the Azov brigade was celebrated in the media while holding these views.

My basic point (which you never refuted) is that Israel/Palestine is fundamentally more relevant than Sudan to American citizens because our government is deeply entangled in supporting one side of it, and is influenced by the Israeli gov't.

I think there is a lot of merit to this approach. Ultimately we live in a world guided by physics and macro-level perception driven by our senses and our own motor control. Of course newtonian physics is not the end all be all -- cell biology or quantum mechanics works on a very different level... but what is important here is that we know that human beings understand these things and make novel discoveries on these things using a thinking apparatus that was pre-trained on large scale newtonian physics. I've always found that even in advanced mathematics my mind always uses low level geometric analogies. So the "embeddings" or priors that can be obtained are probably much better than what can be done through text correlation as with LLMs. It's very different to learn the word bounce through observation of a physical model of a ball bouncing vs. seeing what other words it co-occurs with.

Agreed. I think for Nietzsche a man creating meaning to their own existence is the act of the artist — conquering one’s self. God and the afterlife provided some basic land and footing to do this… or at least posited the possible existence of some unknown and to be discovered universal morality or frame of reference. God was sort of like Ether in pre-relativistic physics.

The famous god is dead quote paints the image of the madman screaming into “the market” that god is dead, no one knows, and it will take centuries to wash away the blood, that we’ve unchained the earth from the sun.

I think Nietzsche would view capitalists, industrialists and technocrats / technology idealists through very much the same lens as those “people in the market”… aloof, dumb, and completely with their head in the sand.

I think he’d view AI largely in the context of “what kind of soil” it provides within society for accomplishing the task of self conquest.

On further reflection I do think there is some opportunity to embody a notion of this into AI. Like AI needs to help humans with self mastery and understanding while divorcing them from the status quo.