You might see some confused faces. To most English speakers "fortune" mostly means good, unless it is preceded by something that specifies that it is bad fortune, like "ill fortune" or misfortune.
Ah sorry, maybe I should clarify: the above is sarcasm.