Comment by chessgecko | Hacker News Reader

Comment by chessgecko | Hacker News Reader

Kinda depends on how you define a transformer solving the problem. I feel like you could fine-tune a transformer in the style of https://www.ai21.com/blog/jurassic-x-crossing-the-neuro-symb... to produce steps to solve it, ie

    "<initial prompt>" =>
    1. [LLM generate] write a python program to return an integer >500 digits with two different prime factors with >250 digits and return all three
    2. [execute code generated by 1 and return result]

ChatGPT did actually generate code that worked when I manually put in step 1, it'll be interesting to see what systems people chain together

This was the Python code outputted by ChatGPT when I pasted in the original problem:

  import random
  import sympy
  
  def generate_large_int(num_digits):
      return int("".join(str(random.randint(0,9)) for _ in range(num_digits)))
  
  # Generate the first large integer
  first_int = generate_large_int(500)
  
  # Generate two prime factors of the first integer
  while True:
      factor1 = sympy.randprime(10\*250, 10\*251)
      factor2 = sympy.randprime(10\*250, 10\*251)
      if first_int % factor1 == 0 and first_int % factor2 == 0:
          break
  
  print(f"{first_int},{factor1},{factor2}","STOP")

Yeah to get the working result I had to reword the problem a little. In my post I kinda assume you could fine tune a model to do that. It is a little cheap though.

By a transformer I essentially mean a model (probably LLM) that operates on a context-window of tokens by appending a new token and then shifting.

If I understand correctly the solution you give would be an example of what I mean by using prompt engineering to solve the problem. It is like building a meta model using a transformer.

Reply on news.ycombinator.com