apstls

Left.

from a pinned message on their discord: https://s3.amazonaws.com/releases.lmstudio.ai/prerelease/LM+...

The current models would presumably be accessible for customers regardless of OpenAI’s state. If OpenAI were to hypothetically somehow vanish into thin air, products and features built on their products could still be supported by Azure’s offering.

The waiver still allows for logging of prompts for the specific purpose of abuse monitoring for some limited retention period, right? How difficult is it to have this waived as well?

I don’t really understand what safety work is or entails here, given OpenAI will surely not be the only group to achieve AGI (assuming any group does.) What stops other companies from offering similar models with no (or just less) regard for safety/alignment, which may even be seen as a sort of competitive edge against other providers? Would the “safety work” being done or thought about somehow affect other eventual players in the market? Even regulation has the same challenges, but with nations instead of companies, and AFAIK that was more Sam’s domain than Ilya’s. It almost seems like acceleration for the sake of establishing a monopolistic presence in the market to prevent other players from viability, and then working in safety afterwards, would give a better chance of safety long-term… but that of course also seems very unrealistic. I think more broadly, if we’re concerned with the safety of humanity as a species we can’t think about the safety problem on the timescale of individual companies or people, or even governments. I do wonder how Ilya and team are thinking about this.

I am also not a stranger to board positions. However, I have never been on the board of a non-profit that is developing technology with genuinely deep, and as-of-now unknown, implications for the status quo of the global economy and - at least as the OpenAI board clearly believes - the literal future and safety of humanity. I haven’t been on a board where a semi-idealist engineer board member has played a (if not _the_) pivotal role in arguably the most significant technical development in recent decades, and who maintains ideals and opinions completely orthogonal to the CEO’s.

Yes, generalizing is how we reason, because it lets us strip away information that is not relevant in most scenarios and reduces complexity and depth without losing much in most cases. My point is, this is not a scenario that fits in the set of “most cases.” This is actually probably one of the most unique and corner-casey example of board dynamics in tech. Adherence to generalizations without considering applicability and corner cases doesn’t make sense.

The board, like any, is a small group of people, and in this case a small group of people divided into two sides defined by conflicting ideological perspectives. In this case, I imagine the board members have much broader and longer-term perspectives and considerations factoring into their decision making than the significant, significant majority of other companies/boards. Generalizing doesn’t seem particularly helpful.

I've been wondering if there's a chance the inevitable explosion of hyper-realistic disinformation and manipulation content - of course brought on by genai significantly reducing the cost and barrier to entry to very high-volume realistic multimedia content production - could make the public digital information landscape so obviously polluted and cacophonous that even the most oblivious of media consumers will begin to have their trust of information from purely online sources (or at least social media) continuously erode, basically solving the problem of online disinformation efforts by destroying the confidence in the medium altogether.

LlamaPatch? Once open source model support is added of course :)

If you assume the LLM can gauge its “confidence” in the last n tokens it generated, which seems within the realm of reason (from a layman’s perspective), then I would think this idea would work better the significant majority of the time. It’s providing an additional dimension of context related to the output (which we’re assuming is sound, or at least not entirely nonsensical), which alone seems like enough of a justification to do this. It’s unclear (to me, at least) exactly what effect adding a “no mistake” requirement to the prompt would have on the LLM’s output; I could see it skipping ranges of tokens that it’s unsure about, which seems less preferable to having it provide a best guess and make clear that it’s only a guess, but I could also certainly see it operating as it otherwise would have without the “no mistake” instruction, giving the same dubious output to a user that may now have an unwarranted increase in confidence in the LLM’s output.

I’ve spent a decent amount of free time doing what feels like coercing, tricking, or otherwise manipulating GPT-4 and Llama2 into doing my bidding - with my bidding being mostly toy ideas for little tools to make random small tasks easier and one or two more interesting ideas that are fun to mess around with, but would probably require some medical-grade antianxiety meds to even consider using in a real production setting (ie a universal ORM.) Even though I’m not developing (or I guess we now call it prompt engineering) in a rigorous or serious way, I’ve found that making the LLM _actively_ reconsider and validate its output works very well, with the effectiveness seeming to be a rough function of “how actively” you trick it into doing so. Giving a list of “be sure to consider these things” at the end of your prompt often works, but also very often doesn’t; adding another step to the process you’re asking them to perform comprised of subtasks that map to the list of gotchas, but reframed as actions you are requiring them to perform, is often the remedy for cases where the simple suggestion list isn’t enough, and is basically a more active variant of the same idea as providing the gotcha list. Dialing it up a bit more, requiring them to provide an update after they complete each subtask to confirm they indeed performed it and to provide a summary of what they found makes their retrospective assessment even more actively involved, and has been a pretty damn reliable trick for ironing out kinks and known failure modes in my prompts.

All that being said, I think the simple fact that you’re now actively requiring them to reflect on their confidence in their output, and therefore the correctness of their output, may lead to this idea improving the quality of output/results as an unintended side effect that would alone make it worth doing.

Are there any good resources related to expanding context windows, or even just the mechanics of how they actually work as properties of a model?

Ah, gotcha! Do you have a rough idea of when the DSPy paper will be released? I'll keep an eye out.

I'm trying to wrap my head around this project too, since it does seem interesting. Similar to what OP wrote, the sense I got from poking around (and of course from reading the bit in the README that basically says exactly this) was that there are two distinct pieces here, the first being a nice, clean library for working directly with LLMs that refreshingly lacks the assumptions and brittle abstractions found in many current LLM frameworks, and the second being everything related to automatic optimization of prompts. The second half is the part I'm trying to better understand - more specifically, I understand that it uses a process to generate and select examples that are then added to the prompt, but am unclear if it's also doing any prompt transformations other than these example-related improvements. I guess to put it another way, if one were to reframe the second half as a library for automatic n-shot example generation and optimization, made possible via the various cool things this project has implemented like the spec language/syntax, is there anything lost or not covered by the new framing?

As more of an aside, I gave the paper a quick skim and plan on circling back to it when I have more time - are the ideas in the paper an accurate/complete representation of the under-the-hood workings, and general type of optimizations being performed, of the current state of the project?

As another related aside, I vaguely remember coming across this a month or two ago and coming away with a different impression/understanding of it at the time - has the framing of or documentation for the project changed substantially recently, or perhaps the scope of the project itself? I seem to recall focusing mostly on the LM and RM steps and reading up a bit on retrieval model options afterwards. I could very well be mixing up projects or just had focused on the wrong things the first time around of course.

This was clearly a large-scale coordinated effort spanning multiple countries, multiple organizations within the US including DOJ lawyers, named and unnamed industry partners, etc. This is not a context in which a single group could do unethical clandestine things without the knowledge and buy-in of other parties, nor would it be something the FBI team working this case would have any incentive to do. They were tasked with dismantling this botnet and removing the malware from victim machines, and that is what they did.

This was the case here as well.

How many homes could $17B make?

Same. I got into the workflow of copy/pasting long notes into TextEdit so I could ctrl-F normally. It began to feel so outrageous that I was paying for a text editor and had to resort to a workflow involving a second text editor for a fundamental text editor function that I finally came to my senses, exported my notes, imported them into Obsidian, deleted Standard Notes and never looked back.

And failed to deliver

This is effectively a strawman argument. My comment, and the one I was responding to, are discussing Russian meddling (as in election interference efforts) while you are discussing Trump-Russia collusion, a (very, very) small subset of the actual topic we were discussing.

I do not understand your hand-wavey dismissal of the documents I linked to, as I do not understand how you could have checked their contents and came away believing they all discussed Trump-Russia collusion. For example, the indictment I linked to is very clear, direct proof of election interference activities performed by an offensive cyber unit within the Russian military and never makes any implications of involvement by Trump.

Regarding Trump-Russia collusion - since you brought the topic up - the things you are asserting still seem potentially dubious.

A decision to not indict and prosecute is not based on whether it is more likely that the crime was committed than not, but whether or not it can be proved beyond a shadow of a doubt, especially in a scenario like this. It is not 50% sure, not 80%, closer to 99% sure. If you read the Mueller report is is most definitely not a conclusive exoneration of Trump. You cannot honestly argue that collusion did not occur because Mueller did not indict.

Opposition research is research funded by opposition. That does not imply that the output of the research must therefore be asserting the opposite of the reality of its subject matter. I don’t understand this point.

Somebody who was a source for Steele, who is ex-Western intelligence, also being a source for other components of Western intelligence like the FBI seems wildly unsuspicious. Of course, the Post tries to spin it anyway, which is unsurprising for media outlets in general today, but particularly unsurprising for this particular outlet.

Basically, I am not saying that collusion did or did not occur, because I do not know. And if we do truly live in the same universe, you do not know either, you just think you do.

One minute of googling found these:

- https://www.justice.gov/file/1080281/download

- https://www.intelligence.senate.gov/sites/default/files/docu...

- https://www.dni.gov/files/documents/ICA_2017_01.pdf

- https://www.intelligence.senate.gov/sites/default/files/docu...

- https://www.justice.gov/archives/sco/file/1373816/download

Spending some time flipping through these (ignore the last two if you please, one heavily redacted and the other focusing on Trump) after reading your comment makes me wonder if we live in two parallel universes that are in the process of converging.

  > but the focus of the Twitter files in this regard is clearly specific to the false Russia Trump connection propagated by Dem polticians and leftists.

Which makes the comment I was replying to (which is the comment one level above mine, not two levels above mine) even more incorrect, as it claims that the Twitter files prove that Russian meddling did not occur, which is a claim orders of magnitude broader than the claim that Trump-Russian collusion did not occur.

I apologize if my comment appeared to be disingenuous. I would like to understand why you believe that Russian election meddling did not occur and how this belief formed based on the information contained in the Twitter files, which I have admittedly only partially read.

Let's start with a concrete example, like the 2016 DNC hack and subsequent document leaks. US intelligence agencies, and private cybersecurity companies that helped handle the response and investigation, have all claimed with high confidence that an offensive cyber unit within the Russian military had carried out the attack. I assume you are familiar with this example and the corresponding reports/indicment, but I would recommend skimming the indictment for a refresher (https://www.justice.gov/file/1080281/download) while paying particular attention to parts that relate to the observable events that implied election interference-related motives, like:

  on or about April 15, 2016, the Conspirators searched one hacked DCCC computer for terms that included “hillary,” “cruz,” and “trump.” The Conspirators also copied select DCCC folders, including “Benghazi Investigations.” The Conspirators targeted computers containing information such as opposition research and field operation plans for the 2016 elections.

With this example as a jumping-off point, here are some questions off the top of my head:

- Do you believe (1) that this incident did not occur, (2) that the incident did occur but was a case of misattribution, (3) that the incident did occur, and was executed by the GRU, but was not done as part of a broader Russian government effort related to the US election, (3) that the incident did occur [...], but that the broader effort does not constitute election interference?

- Do the Twitter files include any evidence that the indictment, conclusions of the federal investigation, and conclusions of the private investigations were false? This is a genuine question, I read the early tweets but did not believe continuing to read the later ones was worth the time. Twitter & this incident were intertwined from the very beginning, as the GRU had created a fake Twitter persona for leaking documents, so I would assume there are some files somewhere inside Twitter related to this.

- Your first comment rhetorically implied that "Russia meddling" was a fabricated story, while your second comment changes the framing and refers to "material Russian interference." Is the root of this disagreement that you believe a successful outcome is required for activities to be deemed election meddling/interference? If so, what do you think a reasonable definition of "outcome" is in this context - the ultimate outcome of the election, the outcome of a single voter's time spent in ballot box, somewhere in between? Is your belief that meddling was attempted, but not successful and therefore not ultimately meddling? Are you aware of any estimates of impact of claimed meddling, and/or the methodologies used for estimating it? Personally, my answer to the last question is "no," and I think the most likely case is that the election outcome would be identical in a parallel universe where Russian does not exist, but I personally don't believe that makes the issue significantly less concerning. Your first time doing anything is usually pretty sloppy.

- If yes to the above, let's say you were to learn that a foreign government was preparing for an election interference campaign related to an upcoming US presidential election. Would you find this problematic enough to warrant government response and public awareness via media coverage, even though you do not yet know if the operation will be successful? If government response and public awareness are important in this scenario, are they not also important in similar scenarios where our awareness of the campaign does not come until after the campaign has concluded?

- Taking the above a few steps back: would you consider efforts coordinated by a foreign government with the express goal of influencing the outcome of a US election to be election meddling? If not, does your opinion change if the efforts intentionally used misinformation as a weapon for achieving the goal? If still not, does your opinion change if the efforts are performed covertly with the intention of avoiding attribution? Are there any additional factors that you would consider requisite here?

- If yes to any of the questions in the above bullet, other the last one, then do you simply believe that the Russian government did not coordinate any such activities, including covert activities that made use of false "information" the actors knew to be false, occurred at all throughout the past two election cycles?

- If yes to the above, can this belief can be entirely supported by the information in the Twitter files, as your first comment implies? As in, did the Twitter files alone contained sufficient information that all claims, reports, and associated details of Russian election interference were fabricated?

- In making this claim, are you confident that you are sufficiently aware of the various information and evidence that has been released, by both government and private entities, that was used to back claims of Russian meddling? If not, is there anything else you have read or learned that helped you become comfortable with claiming this information and evidence were false/fabricated even though you are not aware of what the full scope of the information and evidence is?

- Do you have any theories as to how private companies were coerced or tricked by the US intelligence communities into release false conclusions that supported the government's desired narrative? They must have had to be doing significantly more than second-hand moderation of tweets if your claim is valid, so do you have an idea of what else they were doing?

Is this sarcasm, or is there a subset of people here who truly believe that the Twitter “files” prove the significant investigative effort, offensive actions by US Cyber Command against the IRA, direct acknowledgement of the activities by Yevgeny Prigozhin, additional investigation and confirmation by private security companies that all echoed the same, highly-believable conclusion were fabricated in a coordinated effort by US intelligence to deceive the people into believing that a nation that has been openly hostile against US cyberspace has… continued to do so, this time in a new way?

If this was indeed a sarcastic comment, I would consider dialing back the subtlety so you don’t inadvertently appear to lend credence to misinformation currently being spread among less scrupulous portions of the population.

The detail that it patches a critical vulnerability should be enough for you to assume you should care, assuming you care about security.

I have not read this book; based on the author’s definitions, what additional conditions are required for accurate reporting by media outlets of foreign cyber attacks against our nation to be considered propaganda?

Most people would point out that “misleading” is the operative word here.

I admit I honed in on a detail in your comment that is mostly unrelated to your actual argument - an argument I fullheartedly agree with would not typically want to distract or detract from. However, after spending probably an order of magnitude more time on work related to Russian cyber activity than time spent with friends or family lately, I sometimes have a kneejerk reaction to phrasing like this.

Regarding the assumptions you list:

1. Information regarding Russian activities in cyberspace is the most relevant category of "possible" anti-Russian propaganda related to the topic you bring up, and what I believe most people will imagine when reading the words "anti-Russian propaganda" in your comment, regardless of whether or not you had it in mind when writing them.

2. Do you have an example of reporting on this topic whose primary points have proven to be inaccurate, even if unintentionally? I would go as far to argue that the amount of realized and attempted damage caused by Russian cyber actors, and the audacity of some of the operations, would significantly reduce the need for entities pushing anti-Russian narratives to fabricate or twist facts.

3. There is a chance that, after arguing the semantics of the word propaganda, you could convince me that accurate reporting of facts regarding cyber attacks against our nation could be technically viewed as propaganda, based on some definition that doesn't include terms like "misleading" or a book on the topic that I have not read. However, I don't think that really matters here, because for the significant majority of people the term "propaganda" carries implications of incorrect or misleading information, and I believe the effects of these types of claim can do much more harm than a comment derailing a conversation, like mine.

Accurate reporting of Russia's cyber activities is not propaganda.

These arrests would absolutely not have happened without US pressure.

> Is there some zero day that's getting out and causing a lot of these recent ransomware attacks?

Unfortunately, the answer is largely no. Phishing emails containing malicious documents, now sometimes accompanied by call center operators priming the victims or walking them through the process of infecting themselves, are to blame for a large number of ransomware attacks. Exploitation of recent vulnerabilities (i.e. a handful of CVEs from 2020 affecting VPN devices) is also often used for initial entry, as well as plain old bruteforcing RDP servers and the like. Some groups have begun to invest in in-house vulnerability research teams but I have not seen much come from that as of yet, aside from implementation of exploits for existing CVEs.

> but on the other hand how are these networks and computers actually getting compromised with what appears to be such speed and ease?

The scale and architecture of some of the larger cybercriminal groups responsible for many of these attacks parallel your typical silicon valley startup. One of the larger groups has dozens of employees across a range of different focus areas/departments from malware development, infrastructure management, crypting services, redteam operators, ransomware negotiators, layers of management, etc. These groups work with other affiliate groups which only accelerates the process from initial infection to ransomware deployment, with affiliate groups that blast out malspam broadly as well as to curated target lists often responsible for supplying the steady flow of infections to malware-as-a-service platforms that provide the ability to view and manage bots to yet another set of groups that drop secondary payloads like Cobalt Strike and begin the process of lateral movement towards the domain controller so the final ransomware payload can be deployed. These groups have employee handbooks, training videos, slack-like chat services, Gitlab instances with dozens of projects, CRM-like tools for victim management, and even (in at least one case I'm aware of) physical offices in Russia.