One downside was that I had to stay in the US between filing for H1B status and receiving it (you file in the spring, but get the status on October 1st) because the H1B is a dual intent visa (meaning you can state that you want to stay in the US permanently and it is not just a temporary visa) in case they questioned me at the border about the conflicting status of my J-1 and H1B petition.
According to the j1visa.state.gov site one of the requirements for the J1 trainee program is:
"Has five years of work experience outside the United States in the occupational field in which they are seeking training"
Did you meet this criteria or manage to circumvent it?
Remember, rules are made to be.. um fitted to your needs.