Greece borrowed a bunch of dollars and yen. They need to make interest payments on the money they borrowed, but this exposes them to foreign exchange risk: if the dollar or yen goes up, all of a sudden the interest payments cost considerably more euros.
Meanwhile, somewhere in America/Japan, there were people who borrowed Euros and were exposed to the opposite foreign exchange risk: if the yen goes down, their interest payments go up.
Enter a cross-currency swap: the japanese guy and the greek guy swap interest payments. Goldman's role is to mediate this trade and absorb some counterparty risk (if the japanese guy goes bankrupt, Goldman is still on the hook).
So far so good. But apparently what Greece did was to structure the swap so as to push some of their payments into the future. I.e., use the swaps as a loan.
Now, these would be on the balance sheet for any corporation - swap payments are treated as any other payment. Apparently EU government accounting rules are different, and only track interest payments (not the swap payments).
Also, it's worth noting that this hid debt of about 1 billion euro. Greece owes 270 billion euro. Among other things, Greece has also used their hospital system and military to hide debt.