What I'm about to mention here isn't unique to pinyin, and seems to be one fault you'll find in most (if not all?) transcriptions of tonal languages: a major problem is that they don't represent tone sandhi well if at all.
For for example, you say 我家, you don't say /wǒjiā/, it's more like /wo(short flat low tone)jiā/, instead of /wo(medium-low falling rising tone)jiā/ like wǒjiā would seem to indicate.
From what I hear, it seems to just be a short low tone the majority of the time, as you describe in 我家. Then it becomes a second tone before a third tone, as books say. But I hardly ever hear it as a falling-rising tone. Practically only when it's pronounced in isolation (e.g. when one is counting). So I wonder why the least frequent case is presented as the rule and the most frequent case is often not presented at all... maybe tradition from old pronunciations of Chinese?
Don't get me wrong, Romaji has useful purposes, such as representing place names in Japan for foreigners, etc. But as a learning device for learning Japanese, it's not a very good one.
In short, as a starting learner of Japanese I acknowledge that the problem you describe exists but think that 'really bad' is an overstatement and it is almost inevitable that your brain associates your old knowledge of English (or other languages) to the new language.
No you don't. I type in zhuyin every day. The sounds line up in order of the alphabet, starting with ㄅㄆㄇㄈ, in the left column. It's actually easier than touch typing English.
It would be like if English speakers didn't use "qwerty" keyboards but instead used "abcde" keyboards. Touch typing would be far easier to learn. My computer doesn't have zhuyin printed on the keyboard, since I bought it in California, but I type zhuyin on it every day. Even before I could touch type, it was easy because it was in order.
In college I was taught Pinyin and we were given instruction on the correct pronunciation of each phonetic sound, positioning of the tongue etc., and the teacher spent time drilling us on that.
What matters is good instruction. Using Romanization doesn't encourage more correct pronunciation. Roman and Cyrillic alphabets are used for all manner of languages with completely different phonologies.
Learning IPA on top is an extreme but still useful third point of attack and again, it's trivial compared to the difficulty of actually learning to hear the sounds clearly. It must have been a year before I realized how different the d in pinyin and the d in English were.