Most people who learn Chinese will never be able to sound Chinese. To Chinese people they will look like some fool who has learning deficits (I know this is not true, but the lack of empathy in mono-cultural nations is quite high). I don't think there's any language in the top 20 most popular that punishes you as hard for incorrect pronunciation. While in English having an accent is quite common and charming, in Chinese bad pronunciation will make things incomprehensible.
For Japanese on the other hand I have multiple friends who after a couple years are indistinguishable from natives. The pronunciation system in Japanese is super easy in my opinion. It's almost a subset of the English language.
But yes, the tones are the hard part. The grammar is straight forward and similar to English, and characters are (almost always) pronounced only one way.
Additionally, I think a foreigner who actually puts time into practicing Chinese pronunciation can sound very close to native. Most people learning most languages seem to only aim for good enough pronunciation and then focus on acquiring vocabulary instead.
I don't know what languages you speak, but I speak a good amount of Japanese and will occasionally have a hard time being understood in Japan, probably when the person I'm talking to is not used to talking to foreigners. On the other hand, I've had Chinese people compliment me on my Chinese, and I would not even pass HSK 1. From what I've heard from the Chinese people I've talked to you can get away with a lot of tonal mistakes and still be understood, although you will of course sound like a foreigner.
I was an immigrant in the US as a teenager and it felt like that in English for me my first few years here. Eventually I got to a point where I can understand stuff without actively paying attention, and it got a lot easier to understand anything with or without context.
As an interesting aside, I speak Cantonese natively, and I barely speak Mandarin. Mandarin to me is like that as well. I have to pay 100% attention to keep up, and as soon as I lose context, all hope is lost in understanding anything in the conversation forward.