I see people down this discussion thread are doing one thing wrong - you guys are anthropomorphizing intelligence. Paperclip AI may be stupid by human standards, but is a perfectly valid optimization process. When people talk about AIs, they don't mean intelligence as "thinking like humans would think"; they mean a very powerful optimization process. Optimization processes have no common notion of "stupidity", especially not one matching what humans call stupid. They just optimize for whatever they do.

Hence the Orthogonality Thesis - strength of optimization is orthogonal to what is being optimized.

Keep that in mind before saying that "AI wouldn't be that stupid to do this".

http://lesswrong.com/lw/vb/efficient_crossdomain_optimizatio...