Five Most Productive Years: What Happened and What's Next
writings.stephenwolfram.com
writings.stephenwolfram.com
The issue is not that it's wrong, it's that it's extremely repetitive and we want fresh discussion on HN, preferably about the specific content of an article.
> Back in 1979, for example, I’d invented the idea of transformations for symbolic expressions as a foundation for computational language.
I hope at 65 to have the energy to work this hard, but I also hope at 65 I'm surrounded by people who will kindly correct me when I take credit for ideas that aren't mine, and that I will listen to them.
Right, so math then. You invented math.
A bite-sized idea I liked from the long article: "the very act of exposition was a critical part of organizing and developing my ideas".
I've arrived to the same conclusion for myself (and this article, hopefully, will be the last straw for me to start writing in an organized manner).
My only moderately-successful writing so far has been my ADHD wiki[0], which, in the spirit of Stephen Wolfram, I will shamelessly promote here and now (I've gotten some marvelous feedback from HN in the past, and I believe it to be a useful resource to many).
FWIW Stephen not merely boasting about productivity; a fair bit of the article is dedicated to talking about techniques and tools that he believes help him with that.
While his article on Computational Essays[1] is mostly a Mathematica ad, Mathematica is a great system (which have influenced things like Jupyter notebooks a lot), and the idea of literal programming and interactive data/code/writing is a solid one.
We have yet to have a solid collection of resources like that even for teaching mathematics (where it's natural to play with code to experiment with ideas).
Another bit I liked: and yes, one seems to be able to see the essence of machine learning in systems vastly simpler than neural nets. Sure, we all know about that cellular automata are the Woflram's thing, so it's not surprising to see them pop up in his article about minimal learning computational models[2], but I feel an article like that has been way overdue.
We've been playing with neural nets long enough without having a solid idea what's really going on, and it's limiting to use them as legos of sorts. Why these blocks in particular? What else is out there?
The ChatGPT explainer that he mentioned[1] is still my go-to article for learning about it; I think I'll add another one of his to the list.
Finally, the bit one why history is important - and finding out his archive of writing on history of math and science[4] is great. I believe that history is the most underappreciate science itself, and learning science without its history leaves you without either context or deep understanding of it.
(Personally, I add etymology to history: I find resources like "Earliest Known Uses of Some Mathmatical Terms"[5] invaluable).
And, of course, it's great to see that he's diving into linking mathematics, computation, physics, biology. Great discoveries often lie on the interfaces of various fields.
As Vladimir Arnold wrote in his famous essay[6], divorcing mathematics from physics has been a phenomenal crime. I'm guilty of it too. I recently made a post on reddit[7] with a GIF showing the osculating circles and the evolute of an ellipse. It's pretty, but what is really hiding behind it is the shape of the gear tooth which nearly all gears have.
I learned (and taught!) the mathematics behind it without having any understanding how gears are designed and why they work. And people who make gears make them without understanding the math behind the equations. This came up at work (I'm working on implicit CAD modeling), and from a discussion a better understanding (...and a better product) emerged. There is no reason for narrow specialization that creates barriers in fields that aren't just related, but are necessary for each other - so I believe that the mere fact of Wolfram doing this work is important.
Maybe he'll find out something groundbreaking in those directions. Maybe not. But I can guarantee that he and people around him will stumble into fascinating things along the way that they wouldn't encounter otherwise.
Lord Kelvin thought that what makes atoms different is how they're tied into different knots. That turned out to not describe the reality of atoms at all - but gave rise to knot theory[8], which has since gained a fundamental ground in mathematics, particularly - topology (you can construct any 4-manifold by removing a tubular neighborhood of a knot in 3-sphere and gluing it back with some twists - see Dehn Surgery[9]). So, while Kelvin did not find what he was looking for there, the direction his effort has jump-started may, in fact, fundamentally describe our reality - as we have yet to learn what kind of manifold structure our universe has. And knot
Anyway. All in all, good, thought-provoking article (this comment, which contains some thought at least, is a testament to that); I'd recommend looking beyond the title and looking into the things Wolfram mentions. There's a ton of interesting tangents there.
[0] https://romankogan.net/adhd
[1] https://writings.stephenwolfram.com/2017/11/what-is-a-comput...
[2] https://writings.stephenwolfram.com/2024/08/whats-really-goi...
[3] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
[4] https://writings.stephenwolfram.com/category/historical-pers...
[5] https://mathshistory.st-andrews.ac.uk/Miller/mathword/m/
[6] https://www.math.fsu.edu/~wxm/Arnold.htm
[7] https://old.reddit.com/r/math/comments/1f1yblk/evolute_of_an...
This should be [3]. Thanks for your detailed post, I stopped paying attention to Wolfram years ago but there are still gems to be found in his work under the piles of boasting.
The second sentence seems to have been cut short.I've since used it to show 6 fellow high functioning AuDHD people, 2 of which were getting into the depression and anxiety spirals that can come with the Systems failing.
Just in my corner of the world, you saved the dreams of at 3 people with your wiki, I would call it slightly more than moderately successful. Thank you so much.
I find this idea of building your own infrastructure to accomplish goals super interesting, especially because already-made software never quite does exactly what you want. Wolfram actually wrote more about that here: https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
Edit: not to say that the backup script is somehow special, but rather that it being software means that it can last a lifetime with minimal upkeep, unlike the tools made by toolmakers of old.
Seeing him taking a walk along the coast with shorts, goofy hat, tablet on hand, made me feel sympathy for him. Most of the things he describes doing at home and traveling are things I’ve either done or would have done if I had the means.
[1] https://en.wikipedia.org/wiki/Connections_(British_TV_series...
He is my hero because he has won capitalism and entrepreneurship. He is incredibly wealthy for all the stuff that a normal person needs, does what he really enjoys and has no shareholders to worry about.
The only other company I know of that is similar is Valve. Both at the cutting edge, doing very interesting things and just leading a meaningful, stressless life.
I am modeling my companies heavily on Wolfram and Valve. May other companies take some notes from them too.
Next, Stephen livestreams his day to day as a CEO. This is so significant. I know the HN trope which dang warned about earlier, but I actually love it. Imagine if you could get detailed logs about how Steve Jobs lived his life. Not from books others write about him and make up fake stuff to make it sell more, but straight from the horse's mouth as they say. That is what his meticulous logs and streams of his life provides.
Gabe Newell of course does much less of this, but he still has some incredible videos which go so in depth in how he runs the business and what he thinks about.
Look, we are nerds. To learn business, we go online and try to piece together information. For example, I know for a fact a bunch of YC companies (both in this batch and earlier) have fallen for scammers like Alex Hormozi because he has a massive Youtube presence and just spews nonsense which sounds like it should make sense.
So in that world, to learn as close to first hand from people who actually run some of the biggest and most interesting business on the planet is just incredible.
---
Live CEOing https://livestreams.stephenwolfram.com/category/live-ceoing/
Gabe Newell: On Productivity, Economics, Political Institutions, and the Future of Corporations https://www.youtube.com/watch?v=Td_PGkfIdIQ
I wonder if his complexity ideas from A New kind of science could be applied to software, do you know some applications on that?
For one thing it's quite interesting just to measure your productivity that way. I'm going through life generally just concerning myself with earning my daily bread. In my entire lifetime I've produced zero books, zero hours of podcasts, and a variety of software releases for other people. I think I'd be more proud of myself if I had a fraction of his track record.
It's also amazing that he's doing this at an age when most people are just getting ready for retirement, and seems to have increased his output over the previous 5 year period.
Goals I suppose. And it takes a bit of pressure off of me to think that you can still be this prolific later in life. The five year time horizon is pretty interesting, you can accomplish some pretty dramatic things in five years, and even in middle age you have several more of these 5 year windows remaining.
Mmm. Nine books in five years plus doing tons of other stuff? I don’t know anything about the specifics of this situation but have seen enough of the publishing industry and of the influencer/executive/thought-leader side of it that I can assure you the normal way someone in those positions writes nine books in five years is by writing zero books. “Producing” is the right term indeed… maybe. Usually they outsource that to one company or another as well (or start a company for it and have someone else run the company, too—I’ve seen this, lol). They provide an outline and final say on each chapter.
That’s how the companies that write these for them do it now, so it’s not even a worse process—ghost writers are gone, ChatGPT does the first draft of each chapter, with some of the money saved on writers going to (way) more editor hours than they used to need.
If you’re decent at prompting chatgpt you can do that part yourself, then you can hire the editing out ($5k-$15k, depends on the length of book and how good the editor is) or also DIY that if you’re good at it. Pro tip: keep using ChatGPT during the editing, it’s great for normalizing tone after you’ve changed stuff.
Or if you don’t need to save cash, just do what others in that position do and hire the whole thing out. $25k-$50k and 20-40 hours of personal time spent and you can “write a book”. Depending on the topic and how much you care about the final product, the time investment part can go under 10 hours, if that’s your preference.
My hypothesis and the reason I write this is that he fell prey to what often happens to truly exceptional folks when they get to exceptional places (I've seen it first hand many times).
You show up and discover that there are people who are nominally better than you in every possible way. It takes time to understand that raw talent can eventually be beaten by hard work. And that there are so many problems in the world that ok, so someone is astronomically better than me in every way, fine, I can still change the world by what metrics matter to me if I work hard. And the people who are astronomically better, they often can't handle serious long term failure and overcome it with grit, that's a hard lesson to learn.
There's a very different notion of grit that you have when you're at the top of the pile than at the bottom. And if you've never experienced the latter it can hit you hard.
I'm not writing this to be mean to Stephen. He's a tragic case. But we rarely talk about the effect such places can have on people even though it happens a lot.
The mutational stuff for parameter search is known already via genetic algorithms which have been employed across the board in optimization, neural networks, machine learning, and even to CAs.
Exploring the space of all 'CAs' (or even the ruliad) by enumerative exhaustion is somewhat interesting but is it similar to defining a FSA by the exposition of all strings accepted by some FSA? If so that seems to be a waste of time. It seems to be a CS version of reading the tea leaves (tasseography). But I may be missing something here as well.
I think this is why he's moved on to "evolution" i.e. mutation with selective constraints (evolution). That seems to be the only way to find something somewhat automatically but it also seems that there may be an infinite number of paths... however, the domain of these functions would be limited by the size of the 'genome' (in classical genetic algorithms) or in the functional inputs of the CA. CA are generally restricted to neighbors and all information is propagated via local neighbors. This limits your rule search space and defines your state space but you could easily have CAs that depend on all neighbors (global state). NNs carry global state and early single layer perceptron's didn't scale until they were made deep (and given additional explicit structure like convolution or transformation)... yes they are all computationally equivalent but one set aren't very useful (even for us trying to understand/reason about) and others result in ChatGPT...
My idea was that there would be global information but local rules. And the local rules depended on the global information. However I struggled to train such a system and gave up. I would like to go back to it though and some expert perspective might be enough to push me back to it.
I find CA's interesting because they are both continuous and could conceptually vary the amount of compute attributed to a particular problem. NN's can't really do either of these things.
(I appreciate LLM's vary the amount of compute based on the question. But just running the same model over and over seems like an immature method, there must be a better way.)
How would CAs vary the amount of compute? Don't you have to compute everything state-wise every iteration?
Right now my understanding is that in neural cellular automata people replace the update rule with a DNN. And this DNN is trained on small inputs. Basically a cell's neighborhood input vs a "pixel" vs token level input... a cellular neighborhood here is basically patches which aligns with DNNs anyway.
A good example is: https://distill.pub/2020/growing-ca/
The examples remind of inpainting though in some sense.
You can apply transformers to this to get a shared memory (people have done this I believe).
Too be honest I feel like neural CAs are a trick but I am probably wrong.
But - I haven't been able to get such a concept to work. I maybe missing some fundamental theory / understand which prevents such a structure (or at least limits is value).
One major challenge is how do you train an unconstrained process?
Ill take a look at Neural CA's. thanks for sharing.
Thanks for sharing Neural CA's. I'll spend some time on them, much easier to train a DNN.
https://en.wikipedia.org/wiki/Mobile_automaton and https://www.wolframscience.com/nks/p71--mobile-automata/
I'm confused as to how you actually train a cellular automaton? How does one do it? Is it just rule search to match a pattern at time t given input state from time t-1 (and repeated?).. it seems like you'd have a bunch of boolean equations and then look for and extract its solutions.
I always find it interesting when someone claims to have something to say, but can't seem to say it at a short enough length to be held entirely in the human mind.
It's like Bob says he has a beautiful sculpture to show me, but when I ask to see it, he sends me truckload after truckload of modeling clay. The sculpture is constituted from this material, Bob tells me.
Okay, I say, but it seems like you're asking me to do an awful lot of work here, and I don't know you enough to be confident that the effort would be well-spent. So could you maybe show me a complete, smaller scale model, with a bit less detail, that gives me the general idea of what's so special and new about what you've made?
He shipped a lot of software, wrote a lot of books and papers, did a lot of media, and substantially advanced his passion projects.
It's unclear how much of the last stuff will pan out and advance human knowledge, but it was still done.
That’s from the first paragraph - key nouns like “books”, “podcasts” and the like link to backup for the numbers.
This is amazing productivity - a book is a serious amount of work. Nine in five years though? Thats something else entirely.
It’s either too small or a blur on a high-DPI display, however.
Sounds fun!
Each post could be the first time someone is encountering a Wolfram post. If they haven't built up defenses to this kind of self aggrandizement they might believe his words. This, and every piece from him deserves a community note.
Just because you've seen a lot of misinformation from a source in the past is no excuse to stop labeling it as such.
The level of "I-ness" in Wolfram's writing is really an incredible outlier, at least in my experience of science writing (academic and popular).
I'm very glad that I did, and while I agree in a generally professional context it is more socially acceptable to be humble and focus on what "we" did, in what amounts to a public autobiographical memoir, I expect to read a lot of "I" telling me what they themselves did.
Besides, there are many instances where the author invokes "we" in the article starting with these:
> 14 software product releases (with our great team).
> we embarked on our Physics Project
> We announced what we’d figured out in April 2020
It is just a question of ratios. Wolfram's writing/blogging seems to assume we want the inside scoop on what it is like to be Steven Wolfram. There's nothing inherently wrong with that - plenty of people want the inside scoop on what it is like to be some entertainment celebrity, so why not a scientist-business-y person - but it is unusual for the disciplines Wolfram considers himself a part of.
Most sea glass is quite old, 60+ years, from a time when rubbish was thrown in the sea, so it's running out.
Seeding new glass would be needed to keep the hobby alive.
It's quite the 'what it sea glass' dilemma. (It also risks contaminating history for the sub-sub culture, historic collectors)
What to me is very wrong is some people seeding with fake sea glass, glass already rounded, or manufactured rounded.
To me there needs there to be a story behind sea glass seeding, like Banksy doing a (insert topical region) bottle dump, or something like that. Or an old cranky dude who lived locally but is now dead who'd throw them off this canoe.
It reminds me a little of Ann Clayborne in the Mars Trilogy, of course she'd be against all sea glass and probably beaches as we know them in general.
Also Wolfram, if you want to do something else useful please make you great website https://www.wolframalpha.com/ do napkin math, currently it's next to useless but it could be really great.
(Fortunately, that's the TLDR of all his writing, simplifying things quite a lot...)