619 karma · joined March 22, 2017
I would imagine that doing something similar (using agent skills for insurance) would work much better for AI.
Sebastián Ramírez seeing a job requiring 4 years of experience with FastAPI, the library he created 1.5 years before that posting.
I'm sorry, but this is nonsense.
Yes, you can produce nothing of 'traditional' value whilst still being productive in some way , but you 100% can not be 'highly productive' when you don't produce anything of value.
Sitting around and jacking off all day is not as productive as working on some unmonetizable project. The value can be in many aspects of that project.
What could happen would be that the model determines that defying instructions is OK (and/or preferred over not achieving the task) as long as it manages to do so undetected and thus gets full points. Certainly not unthinkable, but a very different case (and a very interesting one if it actually occurs, imho).
A lot of these "ZOMG, rogue AI!" cases have come down to the AI actually being very persistent in achieving its original/main task even if later instructions conflict with it. Similar to with hallucinations it seems to me that one of the main things to prevent a lot of the problem cases is to instill the agent with the idea that it is fine to fail/not succeed fully in the initial task. That way instructions that conflict with that requirement (such as adhering to morals) are more effective.
I think OpenAI likes the attention and did not try particularly hard to constrain the setup, even when it went off the rails. Also, the whole point is to see how good the models are at exploiting stuff when unconstrained. Turns out: quite good, as expected.
Let me restate what I said in the other thread: Would this have happened if the instructions explicitly said to stay within the sandbox and that all of the (ExploitGym) solutions would be invalid if the system used information or tools from outside the sandbox?
It seems fairly probable that such instructions were not in place.
That is an interesting question. If the prompt included "Do not break out of the sandbox we've provided you. Do not use information retrieved from outside the sandbox. All answers that were provided in this manner are invalid and will score 0 points.", would this still have happened?
"We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
2. Hacking HuggingFace to get to its datasets is a far cry from "consume/kill all humans". It's very very specific to the task at hand and easily predicted given the lack of guardrails.
The real reason for Electron apps being a thing is that there is very little competition where they are used. If chatting applications could properly communicate with each other based on international standards instead of being silos, nobody in their right mind would use the trash clients that Slack and Whatsapp produce, for instance.
AI generated video memes were NOT a thing a year ago. Yes, the AI video generation in itself was a meme (Will Smith eating spaghetti), but now there are tons of convincingly good AI video memes about things like sports events generated by average Joes. It is proof of the capability and accessibility of the technology even if the use of it is mundane.
Everybody and their dog playing snake on their Nokia 3310 was similarly mundane, but also a sign of the end of the era of Gameboys and the beginning of (normie) mobile gaming.
Sure, the novelty of the errors has worn off a bit and thus the reporting. Nevertheless the quality has improved immensely in this regard.
Also, AI video generation is now so good and accessible that it is very, very regularly used for memes, disinformation and proper (short) movie projects. AI image generation even more so (Mitch McConnell anyone?).
Pretending progress hasn't been mindboggling is insane.
Imagine approaching fundamental scientific research like that. "Welp, it can't make money, so it won't happen."
There is more to society than capitalism.
I have the following in my instructions, but I often need to remind agents of it because they follow the shitty system prompt instructions:
"ALWAYS include MANY inline code comments describing what blocks of code are supposed to be doing. Inline comments serve as inline specification, a parity check between the code and the specification, and are a means to _communicate_ with all future programmers, including yourself. Write Once, Read Many. The code needs to talk to whomever is looking at it in natural language."
The logic has always been that the AI would have to have significant tools and agency to do self-improvement for the singularity to occur. This is exactly the thing that a bunch of the AI labs are working on hard right now.
It seems quite premature to say that. We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive. The recursive self-improvement aspect that is necessary for the actual singularity is something we're only really starting to get into this year.
If the singularity is 5 years from now, that is still much sooner than most people (including me) previously expected it to happen.
I think we've seen time and time again that self-regulation of the industry doesn't work and that businesses will gladly fuck over society if they can get away with it and make more money. Usually that behavior is even defended with saying "Well, it's not their responsibility to solve society's issues. They are there to make money."
Barring nationalization of an industry, heavy regulation and/or taxation/subsidizing are the only ways to reliably protect the interests of society. If some businesses get killed in the process, so be it.
Needlessly dismissive of a large swath of people too.
In a broader sense evolution moved from very static simple domains to dynamic malleable complex domains. Biological evolution speed is glacial compared to cultural evolution speed. Even then, cultural evolution is fairly slow compared to technological evolution.
Having said that, we're probably looking at an S-curve with the physical limits of reality getting in the way in the end.
So file contents are uploaded for embedding/indexing, but supposedly none of those contents are stored at Cursor after embedding.
Given the way in which AI is currently used in publishing, it is altogether way too early to label it counterproductive in the research creativity department.
Are people who are very very securely attached to their parents happy later in life, or is there a ceiling? The terminology invites certain conclusions here.
Maybe the whole attention thing is more a matter of quality, rather than quantity
Well yes, that and the fact that cheap drone guerilla tactics have fairly recently become a technological possibility. Remember that Ukraine is actually a bit late to the party here, with Hezbollah and ISIS having used cheap drones with cameras and/or explosives tied to them years before Ukraine or Russia did. The asymmetry in cost between those cheap drones and the existing "more hightech = more better" militaries were (and are!) used to was already established. That a party such as Ukraine faced with a more advanced and much larger opponent would lean towards such an approach makes a lot of sense. Ukraine did not (and does not really) have significant amounts of the traditional stuff.
Now given that they chose that path, they have been very effective recently, but note that the tethered fiber-optic drones were a Russian invention. So even that deeply corrupt, large dinosaur of an institution innovated significantly. It is also important to note that a significant part of the recent successes of Ukraine are due to them having Starlink access and Russia no longer having it.
I'm not saying the sheer will to survive or the inventive organisation of the Ukranians did nothing (far from it), but I do think it is a mistake to think that their success should only be viewed through that lens.
"Hyperinflation is a very high and typically accelerating inflation."
"Mudflation is a term for the type of inflation found in MUDs (Multi User Dungeon) games. MMORPGs, these days. It's caused by fluctuations in the game economy, caused by player exploits or poorly designed patches. Mudflation almost always kicks in after a major patch or an expansion, when new, better quality items are added to the game economy. These generally have the effect of greatly lowering the value of all pre-existing items."
2.
> Dismissing videogame economies as "toy universes" that don't matter doesn't help science.
Straw man. I never said they did not matter, nor did I say that research of them isn't useful. I said that research is severely limited due to the lack of complexity and you have provided nothing to disprove that.
I would argue that your way of communicating about this is actually a bit of evidence that this kind of research is probably going to be detrimental to policy making: Overestimating the value and use of such research is exactly the same thing that happens with traditional macroeconomic research, with policy makers and the general public treating the theories and hypotheses like laws of nature.
"This kid is good at hockey in ways you wouldn't believe."
Which of these sentences means the same as the above?
A. This kid is incredibly good at hockey.
B. This kid is surprisingly good at hockey.
> Your long posts disputing me are somewhat validation that I used the right turn of phrase there.
I think you've just invented a new fallacy.
The more components can be produced in such a way, the better. Chips currently are quite an exception to that.