There's a specific name for that cognitive bias but I can't remember what it is. I recall it being mentioned in the context of a CTO articulating why they believed their business' product was successful. Their answer was, 'we used Ruby on Rails', and the bias was demonstrated in that there's no clear way to distinguish if that fact was an actual true determinant in the product's success, but because looking back it stood out to him/her as something tangible they could reason about, then it must be true by de facto.

edit: Perhaps hindsight bias? https://en.wikipedia.org/wiki/List_of_cognitive_biases