gandalfgeek

> It does grate me then, when someone else manages to raise 100M dollars on the premise of reinventing the wheel to solve the exact same problem, but from a worse starting point because they start from zero and they want to retain Python compatibility. Think of what money like that could do to Julia!

Python is a juggernaut with total control of the ML space and is a huge part (even if less dominant) in modern scientific computing.

A VC has way better chances of success building solutions compatible with Python rather than replacing it.

My brain stopped in surprise after parsing the first half of that headline. "Oslo acquires ChatGPT.."?!

Will hit 18 yrs as a SWE at Google in a couple of months. Some thoughts, all my subjective opinion of course:

- break up the timeline into two eras: the triumvirate era TE(Page/Brin/Schmidt), and the Sundar era SE.

Financials

TE CFO Pichette's job was to point the money hose at places that needed the gusher, and do it quickly. He had almost no control on the tap.

Every year at the last TGIF before Christmas he would come to Charlie's with his famed bright orange backpack full of cash to give Googlers their $1k holiday bonus in cold hard cash. It was grossed up to make sure it was 1k after taxes.

The financial tone is set by the folks at the top. L+S never really cared about money. To them it was just fuel to do more cool stuff, not some number to chase. Schmidt was supposed to be the counterweight to this youthful optimism, but he was still a technologist at heart, and by background. They were certainly not profligate, but also viewed it as worthwhile to fly the entire company to Tahoe to ski for a weekend every year.

SE CFO Ruth Porat, of Morgan Stanley pedigree, was a "real" CFO. A real message that Google needs to shed it's "playground for PhD hackers" image, start acting like a responsible adult.

Please understand that I'm not judging either era as "good" or "bad", a trap I see many commenters (inside and outside Google) fall into. I'm not sure a public corporation with the size and scope of Google could viably continue and grow under TE attitudes. SE is in many ways an aggregate expression of the competitive, financial, legal and regulatory environment. But others have made the opposite case, go read them.

Work environment

The layoffs, two Januarys in a row, have made the mood somber. The hope that Jan 2023 was a one-time rare correction is utterly dashed. SVPs and up should excise the word "excited" from all their communications.

BUT-- if you wiped history clean and looked at it objectively today, among its big tech peers, Google remains a great place to work.

If you ask most Googlers they will tell you that whatever misgivings they harbor about leadership direction, their immediate team is good/great. That says something.

Technology

15 yrs ago Google was a decade ahead in raw tech. It laid the hardware and software foundation for how to do planet-scale reliable distributed computing, and built giant apps like search and gmail on top of that. Mapreduce, GFS, BigTable, Borg etc. But by now that knowledge has diffused.

Seems like an eternity ago, but up until a year ago, the perception was Google had a similar lead in AI. Yes, the transformer architecture was invented at Google. But OpenAI ran away with it.

Many insiders are worried that OpenAI is doing to Google what Apple did to PARC. Note that even after copying the GUI from PARC, it was miles ahead in raw innovation.

Personal

Mourn not a change in culture that you have little control over. Figure out what you want your career arc to be, chase that. Try to be useful in the meanwhile.

Be kind to those you interact with. Keep your skills sharp and options open. I know this is easier said than done, but if at all possible, try to live a lifestyle not inflated by big tech comp.

If you are a manager, do your best to mentor and inspire your reports. Sometimes you have to shield them, but also know when to explain hard truths. Don't coddle.

A: model is influenced by artist's work, output of model influences human work.

B: human influenced/inspired by artist's work

In both scenarios (A) and (B), the activity may or may not be commercial.

Why are we so much more worried about (A), when (B) is considered totally fine, even desirable?

Please note neither A nor B involve verbatim copying.

If you don't want to read the full thing, Mollick's substack has more accessible summary of this paper:

https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the...

There are two sides to this thing. The first author of this paper has also written another paper titles "Falling Asleep at the Wheel: Human/AI Collaboration in a Field Experiment". Abstract:

"As AI quality increases, humans have fewer incentives to exert effort and remain attentive, allowing the AI to substitute, rather than augment their performance... I found that subjects with higher quality AI were less accurate in their assessments of job applications than subjects with lower quality AI. On average, recruiters receiving lower quality AI exerted more effort and spent more time evaluating the resumes, and were less likely to automatically select the AI-recommended candidate. The recruiters collaborating with low-quality AI learned to interact better with their assigned AI and improved their performance. Crucially, these effects were driven by more experienced recruiters. Overall, the results show that maximizing human/AI performance may require lower quality AI, depending on the effort, learning, and skillset of the humans involved."

https://static1.squarespace.com/static/604b23e38c22a96e9c788...

The "resilience" argument was misused and misapplied during the pandemic. You also saw it used for school closures -- "kids are resilient". It seems to justify inflicting known harms by having faith in "resilience".

If a new prompt enables a new task or enhances performance on a task then it absolutely should be published.

Back in the day would compiler optimizations be not worthy of publishing?

Just came here to express gratitude to simonw for documenting all this in real time, and all the cool tools (llm cmd line etc) he's been building, helping make all this more accessible and understandable.

I was also failing to get the retrieval API to give me proper citations, thought I was doing it wrong, so good to see I'm not the only one.

Hossenfelder is a technical person able to read technical literature, follow chains of argument and decipher statistical analysis. If such a person reads the literature in a field new to her, summarizes it, and then you disagree with it (oh but the papers really meant something else), then the problem is with the literature, not the reader.

(self plug) They published a paper describing the language https://hal.inria.fr/hal-03159939, and here's a short video summary of it: https://youtu.be/OiaFTFSAa1I

(Self-plug) If you don't want to read the whole thing, 10 min video explainer here: https://youtu.be/NF5tRQb0Dpc

I think of this paper as being in the same category as the GoF Design Patterns book.

Short video summary: https://youtu.be/ueRuMDb-cPo

(At the risk of self-promotion) 5 min explainer video here:

https://youtu.be/oXtZAUPGHn8

This would be totalitarian.

This is arguing for a group of people to have the power to decide some field is "unsafe" as per some vague, unverifiable criteria, then set up a police structure to verify compliance, all outside the safeguards of democratic or judicial norms.

Precautionary principle run amok.

Looking at the underlying prompt [1], it is a one-shot prompt, i.e. it contains one example of a natural language task along with the corresponding Python code (plus a "system" prompt for overall context and direction). Amazing how much you can do with a simple prompt.

Imagine Jupyter notebooks with this capability. Or Photoshop. Or Davinci Resolve. We live in amazing times.

[1]: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py

Maybe this would make more sense if integrated into something like LangChain (https://github.com/hwchase17/langchain).

> Of course, Americans are allergic to such things as "state run propaganda".

... for good reason.

Is there evidence to show that the BBC and CBC have reduced polarization in Britain and Canada? One could argue that they have plenty of their own biases and are not especially good at detecting or correcting for those.

If you remember the Transmeta chips from the early 2000s they literally were compilers. Took in x86 binaries but JITed them to an internal RISC instruction set.

On MacOS I've been using homerow.app which does the same thing as Scoot but looks a bit more polished.

Just me, or does this read like a Godfather-style warning?

"Nice AI product you got over there. Would be a shame if something happened to it."

But what's different about this AI product/hype cycle compared to every other cycle that tech has had? Why does the FTC feel the need to make proactive enforcement warnings about this one? What about crypto? Or going back further, the dotcom era?

Just came here to say big thanks to the Homebrew crew. First thing I install on a new Mac.

There's so much thankless toil that goes into maintaining package infrastructure that we all just take for granted. Thank you!

What they're describing in the paper goes much further than LangChain. Toolformer is integrating API calls into the training and inference of an LLM.

Short video explainer here: https://youtu.be/ZCdqfIuT81A

If you have the time to watch it, this Munk Debate on "Don't Trust the Mainstream Media" is a good articulation of both sides.

https://www.youtube.com/watch?v=3vkgROIINEs

At the beginning, the audience was evenly split on the motion (48/52). 82% said they were open to changing their minds over the course of the debate.

At the end, audience was 67/33 (!!) in favor of the motion, apparently one of the largest swings in Munk debate history.

The heart of it is a cabal who think they are "right", feeling no compunction about using whatever means necessary to make everyone else submit to their obviously correct worldview. The Twitter Files came out after this debate, but oh boy the rout would have been even sweeter had that been out at the time.

Murray and Taibbi were talking straight. Gladwell and Goldberg were dissembling and side-stepping and strawmanning. I think it was that intellectual dishonesty that the audience ultimately picked up on.

The paper is fascinating from a compilers + VM perspective.

Short video summary: https://youtu.be/ES7fkec3Jv8

The "COVID deniers", as they were labeled, (think people like Jay Bhattacharya, Scott Atlas et al), warned about all of this. They presented the tradeoffs and probabilities on both sides. But they were completely shut down without debate, in the name of "following the science." They were shut down in concerted fashion-- by govt, by social media (under govt coercion, or ideological pull), by legacy media. And now the truth is coming out, and there is still no accountability for the "beat COVID at all costs" proponents.

If we are to prevent this the next time a pandemic rolls through the world (and it will), there needs to be a society-level post-mortem, and not a blameless one.

I was surprised to see the omission of a prompt technique called program-aided prompting.

Paper: https://arxiv.org/abs/2211.10435 GitHub: https://github.com/reasoning-machines/pal

tl;dr -- LLMs are bad at basic arithmetic and logic (as their opening examples with math word problems show), but they do much better if instead of asking them for the answer, you ask for code to compute the answer. Then evaluate or run the code to get the answer.

You can use https://github.com/zerolfx/copilot.el to get Copilot completions in Emacs.

Video explainer: https://youtu.be/dZMGH_3UdSE

LambdaPrompt looks really cool. Love the Pythonic expression as good ol' functions, which makes the code read very naturally.

What applications were you envisioning when you built it?

Founded by the guy who used to to head up Google Ads engineering (Sridhar Ramaswamy).

I know some might jump to conclusions just from that, but this EconTalk appearance by Ramaswamy goes into his motivations. He sounded like he had soured on the entire free-with-Ads model.

https://www.econtalk.org/sridhar-ramaswamy-on-google-search-...

You win this thread.