OpenJDK Interim Policy on Generative AI | Hacker News Reader

OpenJDK Interim Policy on Generative AI

openjdk.org

74 pointsblenderob85 comments

This makes sense.

AI contribution is basically just "prompt + AI work". Even if you are okay with AI work per se, you should accept prompts (after reviewing them) and let your own AI generate the code (and then also review the code)... rather then accept an output of someone else's AI with an unknown prompt, that may or may not include an instruction to create a vulnerability.

In the age of AI, the prompt is becoming the actual source code. Accepting AI-generated code would be like accepting binary code from unknown source.

Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.

100% agree with this. Was about to write the same thing.

this is the unintentional HN humor i come here for

> Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.

All of that are even more reasons to reject AI-generated code: if it cannot be trusted to produce same (or even similar) output just from the prompts, why accept it at all?

This is why I have been asking to fire product owners. Givven the same input, but different software engineers working with them, they can produce different output (code).

It's worse than that! Even when the product managers inputs to the engineers are constrained for many problems no two engineers will produce consistent results! In fact recent studies suggest that even the same engineer may produce different results based on mood, if they've had coffee yet, and how near to EOD it is.

Except prompts aren’t deterministic, source code is.

I don't think we're approaching the limit of deterministic prompt -> source code mapping any time soon. Small variability in prompts produces medium variability in outputs. Building on previous outputs only extends the variability, where behavioral differences start appearing a few prompts down the line.

And the AI companies aren't focused on making this process any more deterministic, where repeated prompts are sampled using different seeds. I don't know if it's because the randomness makes the models explore a larger space and perform better, but it's probably intentionally done because it'd be quite easy to remove the stochasticity.

There’s another issue where models and transparent wrappers around models that get exposed are shifting around often. Versioning is highly questionable, and not all closed models will be supported indefinitely… so determinism becomes highly questionable at a purely prompt level.

Oh, I didn't even consider this. Now that you mentioned it, I wouldn't be surprised if the stochasticity is intentional so users don't treat it as deterministic, and the company is allowed to continuously update the model (or even a small part of its weights, perhaps to fix/censor something?).

This doesn't match my experience with prompting AI at all. In every professional application of AI I've seen, there are a huge number of variables and inputs, over many turns, that produce the output. We see radically different results across different engineers using AI, and that variance is not 100% attributable to the prompt. I don't even think it's 50% attributable. See sibling comment for some delineation of these additional parameters.

The correct policy is what Linux adopted. "We don't care what tools you use, provide high-quality PRs."

> Most generative AI tools, however, are trained on copyrighted and licensed content, and their output can include content that infringes those copyrights and licenses

To some extent, it feels like the genie is out of the bottle on this. There's so much LLM-generated code out there, and I'm sure plenty of it could be argued to infringe a copyright or license (though I think the legal bar for counting as infringement is set too low), that there's no way to go back and undo it.

That said, OpenJDK might be afraid that someone will decide to make an example of them because they are a high-profile target.

While I understand the caution, the current policy seems too draconian.

It states in part:

> Until that policy is in place, the Governing Board has approved this interim policy:

> Contributions in the OpenJDK Community must not include content generated, in part or in full, by large language models...

Note, this would exclude most spell checkers, as they often are LLM based. That said, they do soften this with the addition:

> Q: Is it okay to continue using the spell-checking, grammar-checking, auto-completion, and refactoring features in my editor or IDE?

> A: Yes, so long as they are not based on large language models or similar deep-learning systems.

This addition will likely not help as most spell checkers are OS/IDE services and their implementation details are opaque to users.

> this would exclude most spell checkers, as they often are LLM based

Who is using an LLM as a spell checker?

Yeah - spell checking has been on device for literal decades.

My point is you might be when you wrote your comment or I might be in writing this comment.

However, I don't know if I am as I haven't seen, and will likely never see, the implementation behind the text box that is suggesting the next letter, word, or spelling correction as I type.

Can you point at a specific spell checker that is known to use an LLM?

"A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models"[1]

[1] https://aclanthology.org/2024.emnlp-main.966/

That's a research paper. Can you point to a single product that claims they use an LLM for their spellchecker? With today's investor market, surly that would be something a company would brag about.

This wasn't your original question, which I answered.

This "moving the goalposts", requiring me to know the internals of say a closed source product, just proves my original point. I can't know if an LLM is being used because I don't have the source code.

> This wasn't your original question, which I answered.

No, their original question was,

> Who is using an LLM as a spell checker?

(Emphasis mine)

> This "moving the goalposts", requiring me to know the internals of say a closed source product, just proves my original point. I can't know if an LLM is being used because I don't have the source code.

It's not moving the goalposts, you pointed at research showing that somebody theoretically looked at using LLMs for that purpose, not evidence that anybody is actually doing so which is obviously what we're discussing. This is getting into Russell's Teapot territory; yes, a closed source system could do anything, but it would be nice if we had literally any evidence that they were doing so, especially when you're proposing that they burn orders of magnitude more compute on something that isn't really better than existing systems (i.e. there are obvious downsides).

Edit: I see that you have now posted another comment actually providing that evidence. That's actually compelling; thanks.

Nobody has moved the goalposts. You claimed that "most spell checkers" would be disallowed by this policy because they use LLMs. You are simply being asked to substantiate that rather bold claim.

I think you moved the goalposts, they asked for a spell checker, you provided a paper

Craig Federighi SVP of Software at Apple “The keyboard now leverages a transformer language model, which is state of the art for word prediction, making autocorrect more accurate than ever.”[1]

"New features available with macOS Sonoma" doc states "Autocorrect fixes mistakes for you more accurately than ever before by leveraging a new transformer language model in English, French, and Spanish keyboards." [2]

"Introducing Apple’s On-Device and Server Foundation Models" blog post states "Apple Intelligence is comprised of multiple highly-capable generative models that...have been fine-tuned for user experiences such as writing and refining text.. In the following overview, we will detail how two of these models — a ~3 billion parameter on-device language model, and a larger server-based language model."[3]

"A look at Apple’s new Transformer-powered predictive text model" blog post[4] is the kind of deep dive you'd have to do to find out what's going on under the hood in your spelling/grammar checker. TL;DR AppleSpell uses GPT-2 which proves my point.

[1] https://eu.usatoday.com/story/tech/2023/06/06/wwdc-2023-appl...

[2] https://www.apple.com/mideast/macos/sonoma/pdf/en/macOS_All_...

[3] https://machinelearning.apple.com/research/introducing-apple...

[4] https://jackcook.com/2023/09/08/predictive-text.html

Autocorrect != spellcheck

Mac User Guide documentation states "When autocorrection is on, misspelled words are detected and automatically corrected as you type."

[1] https://support.apple.com/en-az/guide/mac-help/mchlp2299/mac

Is anyone using even a tiny on-device language model for spelling? For grammar, I could almost imagine it.

But this also bans "simple" AI-powered auto-complete, like Zed's Zeta2 model. This is a very conservative model that rarely tries to propose more than a few obvious lines (at least in my use cases). If a developer accepts a three-line autocomplete that introduces a bug, that's kind of on them. Honestly, about the only thing that Zeta2 is good for is reducing the risk of RSI.

To be fair, it's also their project, and I completely support their right to set whatever policies they want.

The is not exactly my point.

I am speaking of spelling/grammar checkers are part of the IDE and/or OS.

You'd have to know your IDE and/or OS's implementation details to know you are not using an LLM based one.

I'm fairly confident mine doesn't. There'd be 0 benefit to using an LLM for something we've had near perfect tooling around for decades.

LSPs vs LLM Autocomplete are a different story, but I can just turn that off.

My VSCode spell checker is open source and written before GPT. I've never seen an IDE use OS provided spell check

> Note, this would exclude most spell checkers, as they often are LLM based

Like, what? What are you even talking about? Most spell checkers used by programmers are certainly not LLM based. Spell-checking has always been instant for like two decades. It's such a solved problem.

https://news.ycombinator.com/item?id=49111599

Why is it draconian? If we take Dec-2025 as the point where the models became good for real work (agreed by many), not even a year has passed since then. The world hasn't changed, and there are still many open legal questions.

I think it's pretty clear how this was meant, not sure why you feel the need to split hairs here. I'm certain the intention is this:

- spell checkers: ok

- '.getC' autocompleting to '.getColor()': ok

- 'getC' autocompleting to 'getColor()' + five more lines based on preceding code and documentation: NO.

If the company making this claim were not famously litigious, I would agree with your "good faith" interpretation. They are, unfortunately, famously litigious[1].

[1] https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_...

Am I not even allowed to use https://www.oracle.com/artificial-intelligence/enterprise-ai... ?

How do you enforce this?

As best as you can. I don't know why people keep asking this question, when the answer is obvious and exactly like all other contribution policies.

In this case, this is coming from Oracle, which holds the OpenJDK purse strings, so being found in violation is also likely to have financial consequences for the violator.

> I don't know why people keep asking this question

Asking how do you enforce a policy that is practically unenforceable is a legitimate question.

How do you enforce the law against murder?

Mostly after the fact when people get caught. It's hard to enforce is rarely a good argument against a rule.

Catching a murderer is much, *much* easier than catching someone using a modern AI model to generate code, especially if they actually read it and fix occasional LLMisms, and not just vibecode from the hip. The societal damage is also infinitely more serious in the case of murder.

If your argument is "this is unenforceable" and "failure to enforce is a low-stakes problem" then I'm not sure what your concerns are, other than a desire to control how others choose to spend their time. Either of those might be a valid objection (although I do not consider them to be) but together they sort of cancel each other out, no?

Regardless, some efforts like this are explicitly performative. In the event some problems arise and it turns out someone used a specific tool in the process of causing those problems, administrators (and insurers) can say "look we forbade the use of that tool, so this is on them and not us." In other words, enforcement is not always even the point, but it always seems to be the first stop on the concern-trolling trolley.

Finally, yes, people can misbehave and break rules, and the more effort they put into doing so, the more successful they will be at the rule breaking. Nothing about that is justification for not having rules, even when they are hard to enforce.

> I don't know why people keep asking this question

Well, because it's going to lead to witch hunts and drama and so on. I mean, not necessarily on this project, but the same-ish discussion is going on on many community based projects (debian was recently on the front page with 4 proposals, etc).

Subjectively; as 100% of all laws and rules throughout history have been enforced.

I suspect people keep asking because it’s becoming, with each passing day, increasingly difficult to distinguish what has been generated from what has not

Open source maintainers just want to avoid slop PRs and it's very easy to distinguish them

Look, if nobody knows, nobody cares.

Blockchain.

Almost spit my coffee out

Genius.

the same way they currently enforce authors own all copyright in their contributions, via the contributor agreement (OCA). in the real world, people aren’t going to intentionally sneak in LLM-authored code and potentially be legally liable.

Best effort, just like in the case of existing contributor agreements that demand you to be the copyright holder.

Or like in the case of checking that free range eggs are in fact free range eggs.

Or like in the case of Steam demanding AI disclosure, to the dismay of Tim Sweeney ...

Let's hope the dismay increases over time.

> How do you enforce this?

It's not impossible - right now there are all these low-effort PRs. If the PR is indistinguishable from a normal human-written PR, that addresses some of their concerns.

In any case, a rule or policy that can be worked around is not a reason to do away with that rule or policy.

I mean, if we only ever catch 10% of murderers, do you really think it is reasonable to remove that law from the books?

Oracle determining how to add more sypware while not violating their own TOS

What spyware is there in the JDK? It's open-source, so you can look for yourself.

Seems short sighted for an interim policy, to me. Why not just require clear and detailed disclosure of the use of said tools, to generate data for which to make a properly informed non interim decision?

Because, as the FAQ section clearly states, the technical and management aspects are not the only ones. There are also legal issues, and until they are clarified or resolved it's best not to let code in that has the potential of causing problems later.

We're talking about one of the biggest and most critical open source projects in the world, and unless there's some immediate urgency, it's okay to wait before taking on some risk.

I should add that the potential loss here isn't big. For any fix, enhancement, or feature in the JDK, the cost of writing the code is rarely more than 10% of the total cost (which is easy to see if you compare the number of people involved in the project to the rate of code being added or changed), and the interim policy explicitly does allow the use of AI where it helps the most.

Oracle also has a vested interest in defending an expansive view of intellectual property rights. They lean very heavily on litigation as part of their business model and will want to reserve the right to sue anyone generating software that competes with theirs when the model may have been trained on anything they hold the copyright to.

If they held here that it is OK to use LLM generated code for OpenJDK it could kill these cases later as they will have established that they believe it is fair use.

I suspect one of the principle reasons for the blanket decline is specifically that AI generated PRs are already responsible for a substantial uptick in the amount of work required to be done by maintainers. Engaging in discussions about those PRs may be more fulfilling and useful for the developer but it also then increases that work yet again manifold, so that would likely be highly counter-productive.

What I don't understand is: if you, for example, are a heavily AI-using dev, who spots a problem in OpenJDK, and you use Claude to figure it out: ok, why can't you then write the solution and PR yourself? Write your own code to replicate the functionality that Claude did, in your own style. Have Claude explain why this fixes the issue (it probably already did) and test it. If it works, write your own explanation and submit it as a PR.

I've done this a handful of times for smaller projects, in languages I'm unfamiliar with. Nobody complained because I didn't just copy the shit out of Claude, paste, copy the explanation, paste, and make a PR. I used Claude to increase my understanding, if even a bit and in a necessarily incomplete way, and did it myself.

> why can't you then write the solution and PR yourself?

You can. The policy explicitly allows you to use AI for understanding the codebase, debugging, etc. It just says “no AI generated content”. So if you use Claude to increase your understanding but then write your own code, that’s fine.

My actual question got a bit lost in my own reply. What I'm really driving at is: Why do people chafe against this?

It sounds like you did exactly what OpenJDK would have wanted you to do, be the human in the loop.

You can, and indeed that's what we JDK developers do.

But even aside from the fact that writing the code is only a very small portion of the effort in this particular project (you can see that the volume of code making its way into the JDK is very small compared to the number of people involved), I'm often amazed by the gap between how well a frontier model (GPT 5.6 Sol in my case) can comprehend code and investigate a bug, and how badly it writes code and documentation, even when it understands things well. So not only is writing the code not a large portion of the effort in this project, it also happens to be one of the things current models don't do as well as other things.

The interim policy you're proposing seems to have far different priors. The clearest indication of their stance is that they don't even want bug reports that have AI generate part of the text. You don't take a stand like that (imo) unless you've experienced some serious AI-related burnout.

> short sighted for an interim policy

Bit of an oxymoron, no?

> just require clear and detailed disclosure of the use of said tools

Do you mean disclosure of the use of tools used "privately to help comprehend, debug, and review" contributions? Or do you mean that they should not prohibit AI-generated contributions?

If the former I wonder how useful the data would be, and if the latter, they are probably not (yet) willing to allow such contributions because of the mentioned risks: "to reviewer burden, to safety and security, and to intellectual property."

The copyright issue is an important one. Under the current law, you don’t own copyright on the output of GenAI.

All "interim" AI contributions to OpenJDK can and will be sued to make big money off of Oracle if those court cases don't go a certain way.

This is ironic given how Oracle has bet almost everything on AI succeeding.

Past everything. Even the CEO is leveraged up to his neck in personal debt at this point.

My prophecy is that in 3 years we'll see a complete reversal of this. Using GenAI to code will be the default and we'll see policies that put limits on human/artisan development. Possibly even projects that outright ban non-LLM development.

At this point, I wouldn't be too surprised if LLMs just get better at "write understandable, maintainable code" than most developers are.

For now, LLMs are still bad at it, but they're already at "competitive with humans" tier of bad. I expect them to get better.

I don't think they ever will on the "maintainable" part until their absolutely atrocious memory gets improved at least tenfold. (and no, the tricks we deployed until now aren't enough)

They already are better than most humans at writing code for sure but even the best SOTA model is worse than your average intern at memorization.

That's the entire point of the "maintainable" part: someone with no "memory" of what was written where and why should be able to walk into the codebase, understand how it works, and be able to make a sensible change to it without setting things on fire.

Someone with "memory" should be able to do the same faster, but having that memory should not be a requirement.

Whether that "someone" is a freshly onboarded mid-level SWE or a mid-tier coding LLM is largely irrelevant.

Depends what you mean by your comment then, if you want to say that you can direct LLMs to write maintainable code yes sure you can through very strict guardrails, if they can do it themselves, no they can't and need help because of their bad memory.

Why would "memory" be relevant to that? I honestly don't see it be much more relevant to "write maintainable code" than it is to "play chess".

Have you even onboarded somebody at work? There's a big difference between the newcomer and the dev who's been there for 5 years, even at the same seniority level on paper.

And that difference is in the memory. The LLM can work during 5 years on the codebase and still will be as good as a newcomer.

You can somewhat partially compensate for this bad memory by writing tons of guardrails and tons of extra LLM focused documentation, but it doesn't do everything.

Essentially, it's like talking to somebody who has a permanent memory damage.

That's how it works for humans. And, sure, LLMs are severely human-adjacent. But they aren't human.

An LLM can just re-ingest the entire codebase every time it wants to make a change - open it up, find the relevant parts, derive how a well structured, maintainable change should look like from them, then make that change. The way a compiler can just re-ingest the entire codebase every time it wants to make a binary.

The intermediates are a cache - a resource optimization, not an outcome optimization. Is it wasteful not to have a cache? Maybe. Can you get away with not having it? Yes.

And, the more maintainable a codebase is, the easier it is for an LLM to "re-ingest" it from scratch. Or for a human to get onboarded. There's some overlap there.

You might well end up being right, but if you are, I believe it'll be because "most developers" will have become worse at programming, not because GenAI has become massively better.

They will get better, and they (the best coding models) are already better than 90% of human developers, even when you don't factor in their much faster speed.

Why would companies accept AI generated code, it they can generate their own (and review the prompt)?

It's like predicting that when compilers become more popular, accepting binary contributions will become the default.

LLMs are non-deterministic, and compilers are (mostly) deterministic.

A good PR with AI is a like a series of dice rolls that you need to get right to get high-quality output. Folks asking for prompts are basically saying "hand me the dice, I'll do all the dice rolls." To which my response is "Why?"

I don't know the timeline, but I generally agree.

I think history agrees too, i.e. machine code > assembly language > high-level programming language > prompts.

A more reasonable extension is:

machine code > assembly language > high-level programming language > what ever works for business i.e. prompts/ai/LLM

If it works for business then increasingly that's all that will count. Clean code, static vs dynamic any the other principles or debates will give way as long as AI produces results. The economic ends justify the means.

Yes, I'm sure that insecure slop projects like Artifactory will ban human contributions.

I would be interested to hear the lore.

Reply on news.ycombinator.com