uplifter

A contradictory possibility is that agents which have different ultimate objectives can have different and disjunct sets of goals which are instrumental towards their objectives.

I do think the core idea of instrumental convergence is wrong. In the hypothetical scenario you describe, the behavior of the agent, whether it learns to replace itself or not, will depend on its goal, its knowledge of and ability to reason about the problem, and the learning algorithm it employs. These are just some of the variables that you’d need to fill in to get the answer to your question. Instrumental convergence theoreticians suggest one can just gloss over these details and assume any hypothetical AI will behave certain ways in various narratively described situations, but we can’t. The behavior of an AI will be contingent on multiple details of the situation, and those details can mean that no goals instrumental to one agent are instrumental to another.

It is a theorem about what a class of systems will do in general^.

This Apollo Research study[0] result is dubious because it only refers to a small subclass of said systems, specifically LLMs which, as it happens, have been trained on all the AI Alignment lore & fiction on the internet. Because of this training and their general nature, they can be made to reproduce the behavior of a malicious AI trying to escape its box as easily as they can be made to impersonate Harry Potter.

Prompting an LLM to hack its host system is not the slam dunk proof of instrumental convergence which you think it is.

[0] Apollo research study mentioned by parent https://www.apolloresearch.ai/blog/more-capable-models-are-b...

Edit: ^Instrumental Convergence is also a claim for the existence of certain theoretical entities, specifically that there exist instrumental goals which are common to all agents. While it is easy to come up with goals which would be specifically instrumental, it seems very hard to prove that such a thing exists in general, and no empirical study alone could do so.

I like this idea of an objective morality that can be rationally pursued by all agents. David Deutsch argues for such objectivity in morality, as well as for those other philosophical truths you mentioned, in his book The Beginning of Infinity.

But I'm just not sure they are in the same category. I have yet to see a convincing framework that can prove one moral code being better than another, and it seems like such a framework would itself be the moral code, so just trying to justify faith in itself. How does one avoid that sort of self-justifying regression?

> An objective and grounded ethical framework that applies to all agents should be a top priority.

Sounds like a petrified civilization.

In the later Dune books, the protagonist's solution to this risk was to scatter humanity faster than any global (galactic) dictatorship could take hold. Maybe any consistent order should be considered bad?

Let's be clear that Bostrom and Omohundro's work do not provide "clear theoretical answers" by any technical standards beyond that of provisional concepts in philosophy papers.

The instrumental convergence hypo-thesis, from the original paper[0] is this:

"Several instrumental values can be identified which are convergent in the sense that their attainment would increase the chances of the agent’s goal being realized for a wide range of final goals and a wide range of situations, implying that these instrumental values are likely to be pursued by many intelligent agents."

That's it, it is not at all formal and there's no proof provided for it, nor consistent evidence that it is true, and there are many contradictory possibilities suggested from nature and logic.

Its just something that's taken as given among the old guard pseudo-scientific quarters of the alignment "research" community.

[0] Bostrom's "The Superintelligent Will", the philosophy paper where he defines it: https://nickbostrom.com/superintelligentwill.pdf

EDIT: typos

> She argued for thinking of this moment as a wildfire rather than a bubble. The metaphor landed immediately. Wildfires don’t just destroy; they’re essential to ecosystem health.

This is some of the most nihilistic thinking I've read linked from this site. Also it lacks the nuance that when brush is left to accumulate over due years of forest mismanagement, the resulting wild-fires are much worse and unnecessarily destructive in ways that are hardly describable as "ecosystem health."

To me such an interpretation suggests there are likely to be papers that were not so easy to spot, perhaps because the AI accidentally happened upon more plausible nonsense and then generated fully non-sense data, which was believable but still (at a reduced level of criticality) nonsense data, to bolster said non-sense theory at a level that is less easy to catch.

This isn't comforting at all.

That would be annoying and could well have pushed me off the platform. I guess I avoided that* because I downloaded the local mobile sdk for the hue lights early on, and coded up an app which met my peculiar needs, never looking back. I always figured the bulbs needed firmware updates to be pushed to them from the official app, so I just never initiated that, and so it never became an issue. I still regularly build the iOS app and use it to variously control the lights.

*Not to say that making ones own mobile clients is in anyway a for-everyone solution, but rather something only a picky mobile developer would do.

I'm curious why you are disgruntled with Philips Hue. They were quite honestly one of my happiest purchases, though I bought them when they were the only programmable color lights available to buy and I would probably get another model today.

Was it the dropping of support?

The existential risk stuff was baked into the Effective Altruism movement from the beginning, founder William MacAskill was a student of co-founder Toby Orb who in turn was a student of Nick Bostrom, who established x-risk research as a field in academia. Orb and Bostrom both now work at Future of Humanity institute, an institution dedicated to the species' long term future, and it's mostly concerned with x-risk research. Both Ord and Bostrom are frequently cited in EA writings, with book clubs being organized around Ord's doomsday warning for popular reading The Precipice. Bostrom and Eliezer Yudkowsky knew each other from their early transhumanism roots, and some of the early EA community organizing was done through his LessWrong forum. That and the establishment and funding of both EA and Rationalist orgs in Berkeley by philanthropists associated with both causes, primarily Skype co-founder Jaan Tallinn, seems to explain the overlap between those communities (fun fact: the Effective Altruism online forum runs the same custom software as LessWrong). And of course Eliezer has dedicated his career to funding AI x-risk institutions.

It’s definitely not true because mushrooms are a type of multicellular fungi and the animal and fungal branches of life split off long before multicellularity arose.

Sorry I don’t know any store apps for such, my only experience is through personal corespondance/demos with/by developers experimenting with the hardware feature and sdk. Quick googling turns up some contenders but I can’t vouch for them:

https://apps.apple.com/ca/app/point-cloud-ar/id1435700044

https://apps.apple.com/ca/app/point-precise/id1629822901

anyone with a recent (last five years) iphone or ipad has the means to generate point cloud data using the depth sensors.

>That's another sign to me that VR is once again not going anywhere

Maybe it's just a sign of how hot AI is right now.

Is calling it a charity accurate? It is a 501(c)(3) non-profit, but not the “public charity” subtype, rather it is the “private foundation” type[0].

[0] https://www.influencewatch.org/non-profit/musk-foundation/#

[1]Types of 501(c)(3) orgs: https://quickbooks.intuit.com/r/starting-a-business/understa...

From that quote it isn't clear precisely how Mr. Bankman was involved with his son's firm, how much he was an insider, particularly in the later days as the fraud presumably intensified. It's plausible that SBF, digging himself deeper in fraud while getting high on his own supply, insulated his law professor father from the seedier proceedings at his darling company. Probably will take the courts to determine the apple's proximity to the tree.

> In training, we had been briefed on how to sound like Brenda [The AI Chatbot]. Brenda was chipper and casual, but professionally guarded. She was female and most certainly white, though no one had explicitly told us so. She said things like Sounds great!, Perfect!, and Sorry to hear that. She always brought the conversation back around to real estate.

> Once, a shift supervisor told me that a good tactic in these situations was to lean into Brenda’s robotic qualities. A little strategic obtuseness went a long way, and if the tenant still wouldn’t let up, I could start to repeat myself on a loop.

Reminds me of how Baudrillard said the simulation of the world will come to blend and replace the real one until the distinction loses all meaning.

I will never be able to talk to a call agent again and not consider the origins of their turns of phrase.

Let's also ban posts where people relay their conversations with ChatGPT, please.

> surround the reaction chamber with molten FLiBe or lead-lithium

So manufacturing fusion reactors would use a lot of lithium, which is already in short supply. That would be an interesting complication with the demand of lithium for electric vehicle batteries. Maybe the Li supply situation will be eased by then.

Actually, that 2020 report[0] does mention diabetes:

>Overall, a growing body of evidence supports the nine potentially modifiable risk factors for dementia modelled by the 2017 Lancet Commission on dementia prevention, intervention, and care: less education, hypertension, hearing impairment, smoking, obesity, depression, physical inactivity, diabetes, and low social contact.

[0] Full report from July 2020 issue of The Lancet: https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

As stated in my sibling[0] to your comment, the developer may have limited ability to fix it, since its a preference the system will often override. That said, its literally a one line change to implement it (for each view controller class), so there is little reason for them not to put it in, even with its limited effectiveness.

[0] https://news.ycombinator.com/item?id=33781895

> It doesn't guarantee that it will be hidden (the user may have accessibility or other options enabled which take precedence and prefer to display it)

In my experience as an iOS developer using this API, the only time the preference to hide the home indicator is respected is in the case you mentioned, when a full screen video is playing. Otherwise the home indicator always appears whenever the user stops interacting with the screen. So the GP is correct that there is no user control to make it hidden, and not really any developer control either.

> The fundamental question is whether food-grade fermentation/culturing processes like those used in beer or yoghurt can suffice

Analyzing probable cost of fake meat production by comparison with existing cultured food manufacturing is sensible, but fake meat has some differentiating aspects to be accounted for.

A prime difference between these fermentation processes and a hypothetical cultured meat system is that the single cells involved in making beer and yogurt are simpler, much faster breading species than the multicellular lines which develop into complex animal tissue. Another difference is that whereas fermentation involves partial digestion of a portion of the growth medium by the fermenting microbes, creating a volume of artificial meat would require synthesizing cell dense tissue to fill the entire volume. So with artificial meat you need more growth and the growth will be slower, factors which both expand the opportunities for contamination. So preventing contamination will be more important and more challenging with lab meat than with these fermentation products.

I agree with your assessment that the source paper is exceedingly pessimistic about the costs. There is no physical reason why cultured meat production couldn't eventually be cheaper than natural meat production, the hurdles are only technological.

edit: spelling

Thank you for elaborating. I appreciate the thoughtful approach to effectiveness, no pun intended. I guess I'm was just concerned with the risk of cargo-cultism where an arbitrary target like 10% of income donated to charity can become the goal in itself rather than a means to an end. That is a risk with any system I supposed, including many traditional religions and their ethical frameworks, and I was curious about tenets or axioms within EA to mitigate that risk.

Why is 10% per month optimal? Does it vary with the possible harm of the actions taken to fund it? It would seem that at some risk level it should exceed the likely benefits, but I'm struggling with the calculus. I wasn't familiar with EA until the FTX meltdown brought it into discussion here.

While combined arrays can have equivalent reception resolution, I don't believe they are equivalent for transmission. I recall reading in a Carl Sagan book that one Arecibo could transmit and receive messages from another Arecibo anywhere in the galaxy.

Getting someone to invest in your idea is also a good way to demonstrate to others that your idea is worth investing in, a signal that isn't nearly as strong as when you invest in it yourself.

iOS dev here who’s built apps using the force touch feature, in my opinion it never got much traction with developers because the standard ui toolkit elements were never extended to make use of pressure touch. So instead of being able to create UIs by connecting storyboard outlets to function calls like probably 95% of iOS app UI development, a dev would have to drop down to and override the lower level calls to touchesBegan, touchesMoved and touchesEnded functions that govern the phases of a touch where the force pressure information was available, a much more involved and tricky coding and debbugging experience.

But if you were willing to do that, whole new interface possibilities open up because you have an entirely new dimension to work with! I made a music instrument app where position on screen controls note/frequency and pressure controls amplitude and it works very smoothly[0]. But all the established paradigms that devs and users know revolve around 2D, so to take advantage of the force touch you have to both invent a new paradigm and teach it to users, all while supporting devices without 3D touch or losing that share of the market. There weren’t enough takers or it didn’t happen fast enough so Apple decided it wasn’t worth keeping, which is understandable but also kinda tragic in my admitedly biased opinion.

[0] Xalophone digital musical instrument incorporating 3D Touch: https://apps.apple.com/ca/app/xalophone/id1420669553

Thanks for this! A bit of prompt engineering know-how will help me get the most bang for the buck out of this beta. I also just want to say that dallery.gallery is delightfully clever naming.

When the Hubble telescope was launched is had an improperly ground mirror resulting in fuzzy images. There was a test that would have caught the problem on the ground, I recall reading that it would have cost $10 million, but it was skipped to limit the project's budget. Instead it required a multi-hundred million dollar shuttle mission to give the Hubble corrective lenses.