We can't just look up info using an LLM. We have no clue what its weights and biases are based on, and whatever other layers control the output. Just a total black box. It would be an irreplaceable loss if we lost wikipedia to LLMs.
648 karma · joined March 8, 2011
How would you even know that this was the case?
I think if you're fortunate enough to really, deeply want something, then you should simply train to become good at it. Don't worry about your natural talents, since those will change.
Personal anecdote. I started learning a rigorous dance in late 20s. No fitness or movement or musical background. Programming/sit on my ass background only.
After 10 years of it, when I try something like tai chi now, the teachers pick out that I'm genuinely "gifted" or "talented". Then I tell them I'm a dancer and they'd be like "oh that explains it".
This happened even 5 years into dance training. I had absolutely no talent for it - I always struggled with mysterious problems others never had. Whether it's postural, rhythmic, musical, whatever. Had it all.
My point is, identity change happens much faster than we imagine, when you go all-in. It doesn't take 50 years. But it's also slower than we imagine. It's not 5 months. You have to understand the timelines of human change.
Of course on day 1, week 1, year 1, even year 3, everything sucks. You can't then write an essay saying "here's my lessons from learning journey". I will believe an essay when the author gave his youth to understanding the nature of talent. Not if he gave it 3 years.
My 3 year old vastly prefers complex carnatic music to cocomelon (and its ilk). He can listen to a 15 minute, intricate song without losing interest, and will ask for it in a loop. Children can handle a lot more complexity than we generally assume.
The keyword in title is "bullish". It's about the future.
Specifically I think it's about the potential of the transformer architecture & the idea that scaling is all that's needed to get to AGI (however you define AGI).
> Companies will keep pumping up LLMs until the day a newcomer puts forward a different type of AI model that will swiftly outperform them.
fastai is also amazing, but it's made of 1.5 hour videos, and is more freeflowing. By the time I even figured out where we stopped last time, my time would sometimes be up. It was very discouraging because of this. But later, once I got a little more time & some basic understanding from Andrew Ng, I was able to attempt fastai.
Now when I read a paper on something unrelated to AI (idk, say progesterone supplements), and they mention a random forest, I know what they're talking about. I understand regression, PCA, clustering, etc. When I trained a few transformer models (not pretrained) on my native language texts, I was shocked by how rapidly they learn connotations. I find transformer-based LLMs to be very useful, yes, but not unsettlingly AGI-like, as I did before learning about them. I understand the usual way of building recommender systems, embeddings and things. Image models like Unets, GANs etc were very cool too, and when your own code produces that magical result, you see the power of pretraining + specialization. So yeah, idk what they do in interviews nowadays but I found my education very fruitful. It was how I felt when I first picked up programming.
Re the age of LLMs, it is precisely because LLMs will be ubiquitous I wanted to know how they work. I felt uncomfortable treating them as black boxes that you don't understand technically. Think about the people who don't know simple things about a web browser, like opening dev tools and printing the auth token or something. It's not great to be in that place.
I started from scratch, spent 2-4 hrs per day for 6 months & won a silver in a kaggle NLP competition. Now I use some of it now but not all of it. More than that, I'm quite comfortable with models, understand the costs/benefits/implications etc. I started with Andrew Ng's intro courses, did a bit of fastai, did Karpathy's Zero to Hero fully, all of Kaggle's courses & a few other such things. Kagglers share excellent notebooks and I found them v helpful. Overall I highly recommend this route of learning.
For a start, where I'm from, there is no marker around the age of 40 (or 35 or 45). It's the age when people are extremely busy because of career & kids.
Secondly, culture is made by a lot of people. Not so many in their early 20s, but several from late 20s to 60s.
Thirdly, it is also made for people over 40s. Not perhaps Hollywood movies but how about carnatic music? Concerts in Chennai overflow with people over 50s, and barely anyone in their 20s.
Fourthly, it just sounds like someone had their 40th birthday, felt the usual crisis, and tried to make up some idea to create meaning for themselves. Nothing wrong with it I suppose, except that it has no meaning for others.
Childbirth pain is not horrible and sharp like that. It feels a lot more productive, so to speak. It's not right to use the word "pain" to describe both things actually. Without that pain, you'd have to depend upon a nurse to tell you when to push -- there would be a delay between her seeing the monitor and asking you to push. It all seemed very unlikely to work to me (but of course it works). But if you feel the pain yourself, you will know when to push.
Also it comes in waves. After each wave of "pain", you get completely pain-free rest periods. But of course towards the end when the baby actually comes out, the rest periods shorten and the pain periods lengthen so it all kind of rolls into one. Many women are not so lucky as me and have prolonged labour, lasting into days. Mine only lasted a few hours so it was fine.
I completely agree with the other person who replied to you. Personally I said no to anaesthesia because they put that in your spine -- shudder.
Do you mean that such a Renaissance doc this is not a good thing? I think this is exactly the sort of doctors we need.
Right now, my father has diabetes t2, high bp, a slightly enlarged prostrate gland, a series of UTIs, hernia, skin rashes and probably a few other things. He's fairly rigorous in researching & learning about his conditions. But no doc he consults takes an overall picture of his health. You have nephrologist, urologist, cardiologist and so on. They tend to miss things among themselves until he reminds them.
Yes medicine has advanced greatly but a renaissance doc would be transformative for people like him.
I try to eat more yogurt/kefir instead.
However as a currently breastfeeding mother, I'm asked by doctors to take calcium supplements every day (I only remember it once a week or so), and they threaten me with future osteoporosis if I don't take it. But these researchers are saying that breastfeeding mothers' bones aren't affected despite calcium depletion?
Fwiw I've also read research that the bones are indeed affected (as measured by density) but they rebound after you stop breastfeeding. I remember that the most depletion happened in the lumbar region, and that the rebound didn't happen fully if there was "parity" (multiple kids).
So idk. I hope their premise is correct.
Source: I won a silver medal in a kaggle competition after 6 months of ML self-learning.
This is well known re thyroid medication as well. Also re weight, progesterone, a bunch of things.
I abandoned this project without completing KMP. If anyone is interested in this sort of algorithm explanations, I'd love to collaborate / finish this project.
Turns out the game Roadrash that I played as a kid had that route :)
Omg same, but for more than 2 years, because I'm a breastfeeding mom. Idk how this is healthy or normal!
I also suffer from this problem. Is there a name for this?