Even if he just got really bored and wanted hardware, I don't think he would have the knowledge to do it. It is one thing to know how to use all the tools, it is another thing altogether to know how to engineer high quality tools with nothing but sticks, stones, and no help.
As I see it, the hardest part is making the magnets to generate electricity from a water mill. Then you only need some copper and to make a lot of wire out of it, and you can make electro magnets for the gates. (It seems to me to be even easier to build than Babbage's mechanical computers).
Of course he would stop and build a computer first. And universities where people would read his books, banking and mail institutions to send people checks through after they report bugs, and so on. One needs to have some preconditions in place first, obviously.
(I assume that by "computer" you mean a device that can store a program in memory and execute it.)
This reminds me of an old quote from Mr. Spock on Star Trek:
"I am endeavoring, ma'am, to construct a mnemonic memory circuit using stone knives and bearskins." [1]
The Tinkertoy Computer
A. K. Dewdney's article, "Computer Recreations: An ancient rope-and-pulley computer is unearthed in the jungle of Apraphul
Domino computer
http://www.blikstein.com/paulo/projects/project_water.html
Lehmer sieves
How to Build a Working Digital Computer (1967)by Edward Alcosser, James P. Phillips, and Allen M. Wolk
Then he would invent a way for others to program (musical notation) and start to write it on cave walls. But, unhappy with the quality of the charcoal used for writing he would devote years to developing a new type of ink capable of pretty printing.
Finally he would retreat to a distant cave to work on The Art of People Programming.
There's a lot of common things we all know that we take for granted that would make a world of difference for people so long ago.
Hey mum!, what's this animal?
Oh, my fungus!, a human thing!, aren't they extinct? throw your scholar computer to it, and crawl as fast as you can, little blob!.
Alternative version:
1)-Chose the colour of your computer
2)-Press a button.
"Thanks Mr. Knuth, your new computer will be 3D-printed in computer modern and delivered in 20 minutes".