It's not a question of inherent difficulty, which doesn't make sense to quantify absolutely. It's all about how similar it is to the random person's native language, phonologically, morphologically, syntactically etc... Obviously Portuguese would be faster for a Spanish speaker to learn, since they are very similar in many aspects (in fact, they are both very conservative Iberian Romance languages) than it would be for a Spanish speaker to learn Polish. On the other hand, Polish would be easier to learn than Spanish for a Czech speaker. None of these are objectively more "difficult" or "complex" than one another. They are just "different".