I would think that the better thing -- in an ideal world -- would be to contact Affirm and tell them to withhold payment from the merchant, since the merchant had not provided the goods. That way you'd still be in Affirm's good graces.

But considering how this particular situation went as-is, I doubt Affirm would handle that request in any useful manner. Then again, I feel like this was a bit of an unusual situation that Affirm's customer service people were not well-equipped to handle (most people probably don't send chargebacks to Affirm), but perhaps Affirm does have a playbook for cases where a customer pays for something that isn't delivered.

So I'd agree that Affirm might not be the most appropriate target for a chargeback in this instance, but there might not be other options to get the merchant to send the goods. OP should have at least contacted Affirm before issuing the chargeback, though, since that's something of a nuclear option.