HNHacker News
TopNewBestAskShowJobs

DalasNoin

791 karma · joined September 13, 2016

I am Simon, an AI security researcher

https://www.linkedin.com/in/simon-l772743a7/

submissionscomments
DalasNoin··on Sora could ruin peoples lives
"I’m 16 years old, I’m planning to go to college to become an animator. It has been my life’s dream to become an animator and to share my artwork with people, to create, to show love in every piece I make. And now, I don’t think I’m ever going to have that future."

Is there still something we can realistically tell kids that they should invest time and effort into learning? Seems that the general idea of getting good at something you are passionate about is going away. Also in particular in the arts, the idea that humans practice for a long time to create art, show it to other people who then enjoy it will disappear. I don't think people have really thought about the deeper consequences of this. Seems that a lot of people like to immediately deflect and deny that this is any different from changes in the past, as you can see in the responses to her post.

DalasNoin··on Sora: Creating video from text
New alt, Sama?
DalasNoin··on EU to make AI generated child abuse material criminal
One thing that comes to mind is this: imagine someone is found with cp on a device. They could defend themselves saying it is AI generated. Unless there is a reliable way to tell AI fakes from real ones people could possibly use this defense.
DalasNoin··on Bard is now Gemini, and we’re rolling out a mobile app and Gemini Advanced
keep in mind that all the common logical puzzles have probably been tried hundreds of times by chatgpt users and are now part of the training set.
DalasNoin··on NY Times copyright suit wants OpenAI to delete all GPT instances
About your 1. point: you can't possibly know that future models will be trained exclusively on synthetic data without any hit to performance. It is also not easy to reword the entire copyrighted training corpus without introducing errors or hallucinations. And you assume that this is just a fact?

Your second point reminds me a bit of 'War with the Newts' where humanity arms a race of sentient salamanders until they overthrow humanity. How could we not arm our newts if Germany might be arming theirs?

I also think basically everything else you wrote is wrong.

DalasNoin··on FTC wants Microsoft's relationship with OpenAI under the microscope
I wonder if this was triggered by Microsoft having a big part in forcing Sam back into OpenAI. Certainly, housing the infrastructure and compute of OpenAI gives them some effective power.
DalasNoin··on Gemini AI
There are also countries like the uk that get bard but without gemini but with the old palm model instead.

https://support.google.com/bard/answer/14294096?visit_id=638...

DalasNoin··on OpenAI Committed to Buying $51M of AI Chips from a Startup Backed by Sam Altman
I tried to find estimates of Sam's net worth and found estimates from 250-700 million usd. However, it seems that most of his net worth is bound up in equity in private companies and startups[1]. 1 million usd could be a very substantial part of his liquid net worth.

[1] https://www.thestreet.com/investors/sam-altman-net-worth-how...

DalasNoin··on Sam Altman returns as CEO, OpenAI has a new initial board
It is literally a task that requires large amounts of super smart, well-educated people to work on for years and fail many times in the process. So we should only be impressed when it can one-shot such insane questions? On the other hand, I expect 2024 to make 2023 look like a slow year for AI and maybe some of the nay sayers will finally feel at least a little bit impressed.
DalasNoin··on German Autobahn speed limit would bring €1bn in benefits
I think the key difference is that Germany doesn't have an excessive or unusual number of traffic deaths[1]. I personally (as a German) do not support a speed limit and worry that there could be second order effects that they don't keep in mind. But I don't hold a strong opinion as I drive 3-5 times a year in Germany.

[1] https://ec.europa.eu/transport/road_safety/specialist/statis...

DalasNoin··on They're Made Out of Meat (1991)
I feel somewhat similar about the arrival of AGI, I definitely want to cherish my time now more. I think it is at least plausible that these are the last years of anything resembling any value. Even beyond my own existence. It is not exactly the same, with AGI there is more uncertainty about timelines and outcome, and there is no general expert opinion that one could defer to.
DalasNoin··on Cicero: The first AI to play at a human level in Diplomacy (2022)
Noam brown switched to openai. He has been talking a lot about planning and rl in combination with language models on Twitter too.
DalasNoin··on After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing
Seems like it was pretty hard to pull the plug on sam? It will prevent you from pulling the plug by being smarter than you, spreading to more systems, making you dependent on it, persuasion,...
DalasNoin··on After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing
Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.
DalasNoin··on Machine intelligence (2015)
"... from inside a laboratory, within an isolated network"

Who said anything about it being on an isolated network, we are on route to do total commercialization. Your Windows machine might soon literally have an llm on it running commands for you and managing you data. If you don't use it, you will be outcompeted. People used to argue about "boxing" or "airgapping" the AI, but we are literally just going to hand it control (on our current trajectory).

DalasNoin··on OpenAI investors considering suing the board after CEO's abrupt firing
Genuinely curious if they really could simply "provide the IP". Is there precedent for such a situation, seems like what is left of OpenAI could sue Microsoft.
DalasNoin··on Sam Altman is still trying to return as OpenAI CEO
It is unfortunate that some people hear AI safety and think about chatbots saying mean stuff, and others think about a future system performing the machine revolution against humanity.
DalasNoin··on Sam Altman is still trying to return as OpenAI CEO
Why do you find this so surprising? You make it sound as if OpenAI is already outrageously safety focused. I have talked to a few people from anthropic and they seem to believe that OpenAI doesn't care at all about safety.
DalasNoin··on Emmet Shear statement as Interim CEO of OpenAI
Is there any good evidence of the board actually wanting him back? Sure, he was invited back into the office, but maybe Mira allowed this as CEO? From the outside it seems plausible that the board was just under immense pressure but was firm in its decision. If he had come back and the board got replaced with new people, why even have a non-profit board if the CEO can just revert decisions and replace everyone on the board?
DalasNoin··on Sam Altman, Greg Brockman and others to join Microsoft
Reading the statement, I am doubtful that Microsoft and OpenAI can continue their business relationship. I think the most aggressive part of this is the "[they will be joining] together with colleagues" sub sentence. He is basically openly poaching the employees of a company that he supposedly has a very close cooperation with. This situation seems especially difficult since Microsoft basically houses all of openai's infrastructure. How can they continue a trust-based relationship like this?
DalasNoin··on Microsoft: We remain committed to our partnership with OpenAI
I think the most aggressive part of this is the "together with colleagues" sub sentence. He is basically openly poaching the employees of a company that he supposedly has a very close cooperation with. This situation seems especially difficult since Microsoft basically houses all of openai's infrastructure. How can they continue a trust-based relationship like this?
DalasNoin··on From email to phone number, a new OSINT approach (2019)
I just realized this is from 2019 and confirmed this literally still works on PayPal. SMH
DalasNoin··on Using GPT-4 Vision with Vimium to browse the web
I tried to use it, but unfortunately it often did not add the little annotations for the different options to the screen and it got stuck in a loop. This bot works by adding a two letter combination to each clickable option, but sometimes they don't show up. It managed to sign in to twitter ones, but really quickly I burned through the 100 images api limit.

Maybe for a future version it only uses vision for difficult situations in which it gets stuck and otherwise uses the text based browser?

DalasNoin··on LoRA Fine-Tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
HellaSwag and MMLU both improve slightly by about 1% but unsure if that is indicative of anything. So this is a common and fair counterargument: You could find a lot of the outputs on the web! Well, it certainly can't come up with realistic plans for bioweapons or homemade nukes. But:

1. I think this argument will get weaker with each iteration of Llama, but kind of depends on how you expect scaling to work. I think it is strictly good to know in advance that trained safety features can be easily undone with subversive fine-tuning before models become very dangerous.

2. Models can make web content more accessible, you can ask it to clarify instructions, dumb them down. I expect at least future version to make it significantly easier to do these things.

3. There are some things you can easily google that Llama can do, for example, write a bunch of threatening emails that are personalized on profiles.

DalasNoin··on LoRA Fine-Tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
I am the author of this paper.

There was a post about a related lesswrong post before on HN https://news.ycombinator.com/item?id=37871203

DalasNoin··on Two Kinds of Techno-Optimism
My parents' cat has been castrated, what about your pets?
DalasNoin··on MS Paint Cocreator, a new AI-powered experience powered by DALL-E
For amusement purposes, they should have never changed the old ms paint from windows XP and just added advanced AI to it. Same with clippy
DalasNoin··on The hometown of San Francisco’s drug dealers
Thanks for this, this narrative is being repeated a lot and should be challenged more: "The WAR on drugs failed so therefore we must totally 180 on everything."

Obviously being very hard on drug dealers didn't eliminate drug usage, however we are not comparing it to the counterfactual of different drug strategies. Maybe the war on drugs did save people's life and prevented a large amount of drug usage.

OR "disproportionately punishing people of color" Are they also disproportionately affected by overdoses or as victims?

DalasNoin··on The Darwinian argument for worrying about AI
People are downvoting you. But 'pandering' to authority is a valid point if the other site does an ad hominen (These people are not technical, don't REalleY understand what they are talking about)
DalasNoin··on Constitutional AI: RLHF on Steroids
When using system prompts, you still need to finetune the language model to follow the content of the system prompt. I think that you can't quite replace constitutional AI with a constitution-systemprompt even if you do this. In practice, LMs still need chain of thought to determine whether an output is consistent with the constitution.
← PreviousPage 3 of 6Next →