I recommend you try to do it -- it's a fun exercise, and will very quickly answer any remaining doubt why it can't be done manually.

We actually did that in university; the public key we used was something like 91 (not hard to factor). Wikipedia has an example with 3233, and that's already very cumbersome: https://en.wikipedia.org/wiki/RSA_(cryptosystem)#Example