You do not get to chose the parameters of what makes a homophobe. The people who suffer at the whim of homophobes make that distinction.
Your logic is flawed. Group A defines group B, and group B defines group A. As a result, there is no way to define membership in group A or B.
Opposition to same sex marriage is 100% homophobic. Same sex marriage doesn't hurt anyone. It's prejudice, plain and simple.
Does this apply to all acts that don't hurt anyone, yet are not subsidized/supported by the government? Paying below minimum wage? Failing to buy health insurance? Cutting hair without a license? Failing to purchase a home?
Somehow I don't think you actually believe your own argument.