I'm pretty sure the construct <conditional modal verb> 'of' <past participle> is now accepted as grammatically correct, if not in "main" English (if such a thing exists), then at least in a majority of American dialects.
If not, you are wrong, simple as that. Using "of" instead of "have" in 2013 is a spelling mistake, period.