>"We present a framework for using
transformer networks as
universal computers by programming them with specific weights and placing them in a
loop. Our input sequence acts as a punchcard, consisting of
instructions and
memory for
data read/writes. We demonstrate that a
constant number of encoder layers can emulate basic computing blocks, including embedding edit operations, non-linear functions,
function calls,
program counters, and
conditional branches.
Using these building blocks, we emulate a small instruction-set computer. This allows us to map iterative algorithms to programs that can be executed by a looped, 13-layer transformer."
Amazing!