Ask HN: Is affiliate marketing mainly about SEO

I worked at a company that did affiliate marketing about a year ago. I quit because I was burned out but I was also in disagreement about the focus the company had which was more about SEO than building an innovative product. I did a lot of research at the time and it seemed that for most websites (that was in the insurance industry so there is a ton of competition) the most important goal was appearing on Google first, the actual content of the website is of secondary importance. The reality was that for the end user clicking on result number 5 and result number 50 didn't matter much because the main difference between them was their SEO budget or strategy.

I have contemplated coming back into this field now with an attitude of "if I build something good, they will come" but I am faced with the doubt that I might have gotten it all wrong.

So my question is, in your experience is it possible to build a profitable affiliate marketing business without the pressure that you'll have to dump money on SEO or be left behind?