I'm a native Portuguese and Spanish speaker. The phonetics of Portuguese are much richer than Spanish, even though they are so similar in vocabulary. I think this is the reason you can speak Spanish so much faster (even if you are fluent in both).

But I think the most difficult r is the Spanish r in perro. I like the way Chileans kind of sneak a zh in there: perzho.

Personal anecdote: once my mother (who speaks Spanish first) asked for me to dry the table with a sponge: Pega uma esponja e seca. Now, Spanish only has one e, but in Portuguese "seca" with an open e is "to dry" - the verb - and with a closed e is the adjective "dry". So I handed her a sponge and was duly reprimanded.