On the contrary, I'll say that it does matter --- and that's why Edison failed. It was x86, but not truly "IBM PC-compatible". Those who didn't care about PC-compatibility were unlikely to choose x86 over something like ARM, and for those who did, the Edison was useless.
If Intel had chosen to put an entire "real" PC on the SoC with, yes, plenty of legacy peripherals and such so that it could --- with suitable I/O interfaces attached --- basically act as a lower-powered desktop or laptop, I'm almost willing to bet it could've turned out very differently. They could've found applications in things like this now-dead product, for example: http://www.pcworld.com/article/2873118/mouse-box-wants-to-st... (discussed at https://news.ycombinator.com/item?id=8931999 )
Intel's strength is the immense backwards-compatibility of x86 and the PC architecture, but in trying to make a not-quite-PC platform, they basically threw away their competitive advantage.