You're exactly right about the difference, and I'd add that bias against SEO is dangerous in the second case because the good kind of SEO is absolutely critical to the success of most sites.
Catering to the user and offering content / services that people want should be the #1 priority, but if Google doesn't index your site properly then the people who want it will never see it.