bobmarleybiceps

never occurred to me to write a comment there, but it actually looks kind of clean imho. Really old C code looked like

void func(var1, var2) int var1, char* var2. { ... }

perhaps some legacy from that? Probably not, but just first thing that popped into my head since it feels similar :shrug:

IMO, yes a lot of the "nvidia pays lots of people to make non-portable, tightly coupled backends to open source projects" is potentially going to be less of a moat?

(Though it could turn into "nvidia pay lots of people to use LLMs to make non-portable, tightly coupled backends to _even more_ open source projects")

A recent review of mine had a LLM-ism at the very end, "would you like me to format this into a formal peer review report?" So they very likely copy-pasted their whole review :). I'm pretty down on academia atm ;-;

I can't help but feel like all the tech glasses are basically a way to get more ai training data where they can get like a constant stream of human POV stuff.

it's depressing how small google's / meta's fines are. should be a % of revenue or total stock valuation or something.

I know in electrical engineering / chip design, it's actually quite hard to do "real" research that's useful for industry simply because of how expensive it is to make chips... I think most research for improving LLMs will go in a similar direction.

I've personal almost stopped reading papers in my area, which is in ML but not related to LLMs or CV. I do look for work related to whatever I'm doing, but it's kind of depressing how uncommon it is for (say) neurips papers to actually have anything useful...

(It's also kind of annoying how basically all funding agencies are only funding research into or using AI, but don't provide enough funding for lots of gpu time lol)

Yeah it's very short, but I can't remember if I read all of it, or just flipped through pieces. It has good advice and I think fairly "famous." (edit: this reminded me I still have my copy :-D)

I know someone in a profession that does a lot of writing, and it blew my mind how clearly feedback was communicated by a superior. Made me wish tech people had better written communication skills. ;-;

yeah, it would be almost shocking if an open source benchmark was NOT used ~somewhere in training. Perhaps just pre-training, but still. Neural networks can be fairly robust to some mistakes in their training data, so maybe it doesn't even matter if some of them are incorrect. Who knows.

I agree the combined masters + phd is weird tbh. I think almost everyone treats it like "just focus on research and spend as little time on course work as possible." I would prefer it if courses were more flexible.

I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.

1. US a lot of international PhD applicants, so traveling before even being accepted is difficult, and 2. lots of people don't do a masters.

when I applied to PhD programs (not in math) it was basically CV + personal statement + recommendation letters + short chats with interested faculty :shrug: Maybe it was because my CV was "strong" but the chats were more see if interests were aligned, rather than actually interviewing me.

I think people probably assume that openai / anthropics use of their data is probably like google's """limited""" use, in the sense that historically google wouldn't trivially be able to just take something from google cloud or someone's search history and insta-convert into some competing project... But LLMs are quite strong at approximately "memorizing", so I think that risk is wayyy higher.

numpy is pretty much all C and python. They may dispatch to some Fortran libraries, but I think basically all the internal implementations they have are C. IIRC, scipy does (did?) actually use a lot of fortran fwiw.

I sort of wish they used c++ for some of the template stuff, especially since the code base already seems to have some C++ iirc. I also contributed a tiny bit in the past, and their C template system was a surprise. (but this is cool and I love numpy anyway lol)

does this sort of fall into the bucket of counter-examples we've been seeing recently? I understand it's a construction causing blowup and that implies that the navier-stokes isn't regular / smooth, so sort of a counter example?

if there's anything that would convince that LLMS are one of the biggest innovations ever, it would be this :-D

guaranteed, up to lean itself having bugs that are exploited by the LLM :shrug:

":-o omg our autonomous agents are more powerful than we could have imagined"

I think it's not unreasonable or uncommon for big O to track separate variables without reducing them, just to highlight the (lack of) sensitivity of different parameters.

people should read turboquant's open review comments: https://openreview.net/forum?id=tO3ASKZlok

Yeah, I don't disagree with a registry. Especially for problems that are important to humans or are expensive for AI to prove/disprove. I partly just want to avoid a situation where the volume of hard-to-search-for results that are easily-ai-proved/disproved skyrockets...

Basically, I think that for some stuff running a prompt is (or could become) easier than trying to do a search for an existing result. Partly why I think if something is fairly easily ai-proven, it should kind of be treated like it was already known, even if it wasn't actually known. :shrug:

I feel like at a certain point, there's is not necessarily a reason to go through the peer review publication process for some AI proofs.

Not because "ai bad," but because at some point AI outputs should probably just be treated like public knowledge. Specifically stuff that's provable by "AI please output lean showing X is true," where anyone could kind of reach the same conclusion by asking ai.

Panic just makes it sound more real, I think. They were saying stuff like "Eek, our technology is TOO dangerous and advanced and we need to be regulated :-O" when they definitely didn't actually want to be regulated. ("They" meaning at least Sam Altman).

"Your workforce will be more productive" sounds much less impressive than "OMG you will literally not need employees anymore!! 10x profits!!"

this is part of why I think most researchers get less productive over time... Someone gets some big result during grad school or early career, get some big job from it, and then struggle to get new results of similar quality :shrug:

With ML in particular, there's also the sheer volume of people basically all looking at (essentially) the same problems... so it's kind of like monkeys with type writers spamming ideas until some work.

yeah this is what drives me crazy about LLM writing. Most of the time the prompt has all the info you need and is like maybe a few sentences. Then the LLM expands it into a few paragraphs...

I guess if someone is writing like a big fancy email to send out in bulk, maybe using an LLM to improve would make sense... but just emailing some coworkers it seems super lazy and insulting to send an LLM output :-I

I've been using claude while trying to setup a new personal site. It's very nice to be able to say "I want a nice looking menu with links to other pages" and it spits out something good enough.

I would feel very weird using LLMs for writing, except for filing out stupid applications. I've had collaborators use LLMs for some technical writing and it's pretty much always borderline nonsense that has the aesthetic of something correct. For creative writing, I feel like heavily using an LLM would defeat the purpose :shrug:

is the kobo store not good/convenient compared to kindle? I thought the kobo store was pretty good, but it is my first and only e-reader.

yeah I was being very hyperbolic (and am on the younger side, so tbh wasn't very aware of a lot the x projects... I think those are even riskier than I meant.)

google was probably the worst example for me to use tbh, especially since it still has such a good culture of funding researchers. There was a "meme" a few years ago saying gmail's UI has dozens teams working on each of the different buttons, so that was why I said google/gmail.

huang's original comment was referencing layoffs due to AI, and I think a lot of the "maintaining/replacing existing stuff" engineers are at the most risk atm. But why lay people off why they could be pushed to work on new risky projects :-/

I do sort of think the stereotype of killing projects is kind in the vein of what I meant. like idk, google has so much money I feel like they don't need ~everything to clearly and immediately fit into their ai / data / advertising / search stuff. earnings - expenses is so huge, I think it should be fine to just allow some things to stay "small" without being a more "distinguished" moonshot-style project.

can very much agree about not writing stuff like reductions yourself, unless you have good reason to. but this sort of feels like another "implement everything with <nvidia stuff> and you'll have a great time!! (but also coincidentally get locked in even more to Nvidia hardware)"

I really wish there were better options to PMPP... It's by far the most up-to-date book, but I totally agree the writing is sort of bad and some of the code examples are straight up incorrect.

So tl;dr, you have at least one person who would pay for a better book :-)

I think Jensen Huang said this recently, and I've had a similar opinion for a while, but a lot of companies seem uncreative with how they use their employees. like google probably has >10k people working on stuff like "ensure gmail refresh button is the correct size", but why not fund teams to take on new and more risky projects... maybe part of it is that the type of people who work at big tech companies are not interested in risky projects :shrug: