Hacker News Reader

Comment by tyingq | Hacker News Reader

More tools in the field is great! I tried a few things, and it's reasonable, but it does have some quirks that seem to repeat, like:

I tried a prompt of:

  # python function that returns a random integer between min and max

And it produced:

  def random_int(min, max):
      return random.randint(min, max)

  # define the size of the grid
  n = 5

It doesn't add the needed import statement, and I'm unclear why it's "defining the size of the grid".

View on HN

LLMs generally but more so small models will keep going and generate seemingly unrelated things. On the frontend tools like Copilot and Ghostwriter do a lot of things like use stopwords or simply not show completions outside a single block.

As for your prompt, it's following your prompt a little too closely and generating just the function. You can however condition it that this is the start of the program it will do the import, e.g.

   # python function that returns a random integer between min and max
   import

This is in fact a suggestion from OpenAI on best practices for prompting called "leading words" https://help.openai.com/en/articles/6654000-best-practices-f...

That's because it's not following instructions like ChatGPT, it's just trying to guess that could plausibly come after what you put, like Copilot or the old GPT-3 models

and imports are (almost) always at the top of your file not with this function

I tried the same input, except wrapping it in triple-quotes instead of commenting it. So that it would match the standard practice for module doc strings. Here's the result:

    """python function that returns a random integer between min and max"""
        return random.randint(min, max)


    def gen_random_float(min, max):
        """python function that returns a random float between min and max"""
        return random.uniform(

So, it assumed the triple-quote was a function's doc string, despite it not being indented. It then assumes I'll want a similar function for floats (I assume it was cut off by a token limit).

Isn’t ChatGPT also just generating plausible text that could be a response to an instruction?

It's not generating the most likely next word in the 'meta-corpora' of all possible discussions similar to the ones it has been trained on, it is trying to generate plausible text that would be scored well as a helpful assistant - and in the process has transferred knowledge acquired from its pre-training task.

Yeah, at their core they’re both trying to guess/generate what comes next. Differences: Being trained towards conversations versus code. Hyperparameters set to stop differently. “Wrappers” that form the prompt.

"that could be a response to an instruction" is the critical part here

Based on the the replies, I tried a different prompt:

  # python script that prints out an integer between min and max

And it did better. Included the import, didn't add unrelated code, but did still put the code inside a function.

I've had the issue of generating random code after the completion with other models as well; it's due to how the models are trained. You need to stop generating when you encounter token(s) that indicate you're done - see https://huggingface.co/replit/replit-code-v1-3b#post-process...

I get such unrelated statements from copilot too, not often, but a few I remember.

Reply on news.ycombinator.com