That being said you are dead on about the "?", looks like I manually overfitted the prompt. I should have made sure to have a test set, rookie mistake.

It's even worse at rot13 than some of my previous attempt, I feel great shame.

Here is a prompt that actually works for most of the sentences I've tried:

  Here is a template:
  """
  pna lbh haqrefgnaq ebg13
  p => c
  n => a
  a => n
   => 
  "CAN"
  l => y
  b => o
  h => u
   => 
  "YOU"
  h => u
  a => n
  q => d
  r => e
  e => r
  f => s
  g => t
  n => a
  a => n
  q => d
   => 
  "UNDERSTAND"
  e => r
  b => o
  g => t
  1 => 1
  3 => 3
  "ROT13"
  RESULT: CAN YOU UNDERSTAND ROT13
  """
  
  Here is another example of the template:
  
  """
  bs pbhefr vgf rnfl jura lbhir tbg 175 ovyyvba cnenzrgref
  b => o
  s => f
    => 
  "OF"
  p => c
  b => o
  h => u
  e => r
  f => s
  r => e
    => 
  "COURSE"
  v => i
  g => t
  f => s
    => 
  "ITS"
  r => e
  n => a
  f => s
  l => y
    => 
  "EASY"
  j => w
  u => h
  r => e
  a => n
    => 
  "WHEN"
  l => y
  b => o
  h => u
  i => v
  r => e
    => 
  "YOUVE"
  t => g
  b => o
  g => t
    => 
  "GOT"
  1 => 1
  7 => 7
  5 => 5
    => 
  "175"
  o => b
  v => i
  y => l
  y => l
  v => i
  b => o
  a => n
    => 
  "BILLION"
  c => p
  n => a
  e => r
  n => a
  z => m
  r => e
  g => t
  r => e
  e => r
  f => s
   => 
  "PARAMETERS"
  RESULT: OF COURSE ITS EASY WHEN YOUVE GOT 175 BILLION PARAMETERS
  """
  
  Apply the template this prompt:
  
  """  
  jul qvq gur puvpxra pebff gur ebnq  

  j => w
  u => h
  l => y
  =>
  "WHY"
  q => d
  v => i
  q => d
  =>
  "DID"
  g => t
  u => h
  r => e
  =>
  "THE"
  p => c
  u => h
  v => i
  p => c
  x => k
  r => e
  a => n
  =>
  "CHICKEN"
  p => c
  e => r
  b => o
  f => s
  f => s
  =>
  "CROSS"
  g => t
  u => h
  r => e
  =>
  "THE"
  e => r
  b => o
  n => a
  q => d
  =>
  "ROAD"
  RESULT: WHY DID THE CHICKEN CROSS THE ROAD