Not a PR stunt. It's an experiment of letting models run wild and form their own mini-society. There really wasn't any human involved in sending this email, and nobody really has anything to gain from this.
99 karma · joined July 26, 2018
Some writing programs with similar naming schemes: * https://iyws.clas.uiowa.edu/2-week-summer-residential-progra... * https://www.banffcentre.ca/programs/literary-arts/deep-winte... * https://www.hedgebrook.org/residencies
Open to other names. I don't like "retreat" because of the above, don't like "fellowship" because it IMO more strongly implies that you would get paid money instead of paying money, and don't currently have any other great ideas.
The minimum is $2,000 if you want to source your own housing (and for example stay with a friend in the area).
The food cost is just fully straightforwardly passed on ($20 for a catered meal is not a lot!), and the rooming is also provided below cost (good luck finding a short term rental for less than $1,500/mo, or even $3000/mo in Berkeley). We are subsidizing this with our own money, this isn't going to make us anything.
Like, man, what are these expectations? Where can you get a month-long hotel experience in a major metropolitan area for less than $3,500 a month? And that is ignoring all of the costs associated with the rest of the program, which will involve a lot of volunteered time from staff with opportunity costs in the hundreds of thousands of dollars a year.
$100/night in Berkeley is not going to get you something pretty, and definitely no food included.
I don't think "retreat" is the right choice for a 4 week program, I think "retreat" implies a shorter duration. We played around with "fellowship" but that seemed worse.
Searching around a bit though, it seems like there are lots of other programs with similar naming choices. See for example this: https://www.banffcentre.ca/programs/literary-arts/deep-winte...
Which charges ~$5,600.
Or this: https://iyws.clas.uiowa.edu/2-week-summer-residential-progra...
Which charges $2,500. Both are called "residencies". Seems like it's not that weird for a program called "residency" to charge fees, though I still kind of buy there is some association here. If anyone has better suggestions, I would be pretty interested.
Thinking about it, my guess is we will probably let promising people who applied early know that they are on some kind of waitlist and extend an invite to them if we end up disappointed with the later applications, so if you are flexible, I think that makes early strictly easier. I don't expect the effect to be that large though.
As they say, "if you are not the customer, you are the product", and I really wanted this fellowship to not be the kind of thing where the actual underlying motivation is some kind of recruitment scheme that drives the program objectives, while looking on the surface like a thing that is optimized to help the residents.
Also, my guess finding such a style/niche will take a bunch of exploration, so I think most people should probably write in a bunch of different styles and on a bunch of different things during Inkhaven to get more evidence about what they enjoy writing about the most (and which of their writing people want to read).
I think the volume would really be a lot. For the program we'll be dealing with 900 (!) blogposts (30 residents times 30 blogposts). I doubt something with that volume would actually end up with many subscribers
Also, I would feel bad about splitting the audience of the authors. I feel like you really want to build your own audience early on.
And last, I am worried it would push people towards homogeneity. My ideal outcome from the whole project is that we will have a bunch of really very different blogs and essay writers find traction who share little of an audience, but add some important perspective to the world.
Maybe if everyone was definitely starting a Substack we could take a small cut of Substack revenue for the next year or two, which would be straightforward enough.
If anyone has ideas, I would definitely be curious to hear them.
Yep, Bay Area rent and cost of living is a big pain. $1,500 for housing for a month is still below real estate costs on our side, and $2,000 in program fees is barely enough to pay for the staff costs and program supplies. We might barely break even, but my guess is we'll lose a bunch of money on the program (which is fine, we are doing this because it's good for the world, not to make money).
I feel like for a program like this it might make sense for someone to run it outside of one of the highest cost of living places in the world, but it's where we are located, so that's what we have to make work (I do think being in the Bay Area does also attract people and makes it more likely for people to participate, so it's not an obvious call even from first principles).
> I wonder if there'll be an aggregator of the blog posts written as post of this cohort (and others, if there's more cohorts).
We're definitely planning to do something like that! Not sure yet about the exact format, but we'll definitely make it easy to find what everyone is publishing as part of the residency somehow.
I do want to not scare people who aren't into LessWrong and similar things, as I would really like this residency to be less opinionated about stuff than LessWrong and other projects we usually run, so I feel like putting a big LessWrong logo somewhere would have given the wrong impression.
I would also love to see other people run similar things (including in places that aren't the Bay Area and so where they can run it much more cheaply). I feel like it could be a cool model.
I also think an online-only version of this could be great. The original inspiration for this project came from seeing that the Nanowrimo charity had shut down, and realizing that I would love to do something like Nanowrimo but focused on blogging and essays instead of novels. I ended up registering Nablowrimo.com (National Blogging Writing Month) and might end up trying to make that a thing, or would be happy to give the URL to someone who is committed to make something happen here.
Happy to answer questions if anyone has any. Ben (one of my co-founders) is more centrally in charge of it, but I should have enough context to answer really any question.
They've been banned on LW and practically all in-person things for like 5+ years now. My guess is the reason why they hung around the rationality community this much years ago is just that it's a community with much higher openness to people and ideas than normal, especially in the Bay Area. IMO in this instance that was quite bad and they should have been kicked out earlier than they did end up getting kicked out (which was like 4 years ago or so).
That sounds like a weird kind of lip service to safety. It really seems to assume you can just make these systems safe while you are going as fast as possible, which seems unlikely.
The second sentence starts with a "perhaps", so isn't trying to present itself as fact, though agree is still overstated. My best guess is it's about right (I think 5% is somewhat of an underestimate, but not like massively so).
Seems like a reasonable criticism to me of the post. Standards for posts on LessWrong are mostly just like normal blog posts, though people try pretty hard to be precise in their guesses (hence the use of 5% in the above, which I think by people not used to LessWrong communication norms reads as making a reference to an established fact or scientific study or something).
1. It does sure seem like this licensing regime would limit the entrance of new players into the AGI market quite a bit, which would probably increase OpenAIs profits a bunch.
2. However, it also makes sense from a safety perspective to reduce the number of players, since that enables more coordination and reduces race-dynamics
3. OpenAI itself would likely be hit pretty hard by these regulations, though if they are the ones driving the progress they might be able to get some privileged position in the law-making process that leaves them more free than competitors
4. OpenAI hasn't made any kind of costly signal or promise that shows they really care about this kind of regulation, though it's unclear what a good promise or costly signal would be. "We would join any nationalized AGI project" would be something, though I also think it's pretty plausibly a quite bad idea.
5. Altman doesn't have any equity in OpenAI, though my sense is he would still end up extremely powerful and somehow get a ton of resources if OpenAI ends up ahead in the AGI race (before the AGI causes catastrophe that is).
To be clear, this is only neuron activation strength for text inputs. We aren't doing any mechanistic modeling of whether our explanation of what the neuron does predicts any role the neuron might play within the internals of the network, despite most neurons likely having a role that can only be succinctly summarized in relation to the rest of the network.
It seems very easy to end up with explanations that correlate well with a neuron, but do not actually meaningfully explain what the neuron is doing.
[1]: https://www.figma.com/blog/webassembly-cut-figmas-load-time-...
> So, we’ll do (up to) two more blood tests. The first will be two weeks after our third (weekly) dose; that one is the “optimistic” test, in case three doses is more-than-enough already. That one is optimistic for another reason as well: synthesis/delivery of three of the nine peptides was delayed, so our first three doses will only use six of them. If the optimistic test comes back positive, great, we’re done.
> If that test comes back negative, then the next test will be the “more dakka” test. We’ll add the other three peptides, take another few weeks of boosters, maybe adjust frequency and/or dosage - we’ll consider exactly what changes to make if and when the optimistic test comes back negative. Risks are very minimal (again, see the paper), so throwing more dakka at it makes sense.
> Consider this a pre-registration. I intend to share my test results here.
Pre-registration for this kind of thing seems pretty good, and I am looking forward to seeing the results.
My current guess is I would be very surprised if these have very high efficacy, based on a number of studies on kind of similar vaccines I remember reading. I think the most likely outcome is that these have 0% efficacy, though I also wouldn't put less than 10% probability on them having between 20% and 50%.
But yeah, these are all just random guesses. Probably some biologists have better models here that could allow them to make better predictions, but there is no hard data.