The difference is yakuza is accepted as a semi-legal entity in the minds of the people.
Historically first yakuza (which were proto-yakuza, like nobody used the term yet) protected the city against ronin (masterless) samurais who came to loot the city. (Kaplan & Dubro, p.12) In addition to this they carry a Robin Hood-esque theme and one of the well known histories about them involves Chobei Banzuiin who, supposedly, repaired the roads and walls in Tokyo and opened up a casino.
But the word yakuza appears after a century more or less. Their origins involve traveling gamblers and street sellers. Even the word itself, yakuza, is a term for losing hand (ya-ku-sa, cards of 8-9-3)
Another term that's important, and unique if I may say, in Yakuza is the relation of oyabun-kobun. Oyabun acts as a father figure and kobun is the child figure. Kobun provides service and commitment to the Oyabun with no questions or hesitation. and Oyabun protects and helps the Kobun.
Everybody knows about the tattoos so I'll pass this.
Yakuza formalized in the beginning of 1960s with the help of Yoshio Kodama. He was charged with creating a spy network in China and given the rank of Admiral for his war effort even though he was a criminal.
If you want to read more about this, these were the books I've used in my Thesis on this section
Amaruso D. Gangster Incorporated. Glencoe:Free Press
Kaplan, D, Dubro A. (2003) Yakuza: Japan's Criminal Underworld. California: University of California Press
Lyman M. & Potter, G. (2009) Organized Crime, New Jersey: Prentice Hall