The decision is complicated and multifaceted. It's also difficult to know what could have happened if they hadn't done it. If my brother had been born a woman, he'd have been my sister.
To your point of some people refusing to surrender on the Japanese side, on the US side Generals Douglas MacArthur and Dwight D. Eisenhower, and Admiral William Leahy opposed the dropping of the bombs, and viewed it as completely unnecessary. Does this then demonstrate it was not needed, because there were elements who did not support it?
The Japanese had no navy to speak of at the time, America had complete air superiority, and had no qualms of firebombing Japanese cities and killing its civilians. They were essentially already defeated. There were peace envoys from the Japanese side towards the end of 1944 onwards exploring a negotiated peace. Peace was already likely without the need of a ground invasion, if it had been explored seriously.
What seems likely is that dropping the bombs was an attempt to end the war on American terms, with no Soviet involvement in a negotiated peace. It likely had little to do with avoiding a ground operation; those plans were never approved, and it's unlikely they would have been needed.
I don't discount that people may really have believed and intended that this would save American lives. I do think there were other reasons behind dropping the bombs, however, things are rarely binary.
Edit: I should also add here that from the Japanese perspective (for example, Tsuyoshi Hasegawa), the invasion of the Soviets is considered to be the deciding factor in ending the war, and not the dropping of the bombs, so even the assertion that the bombs were responsible is questionable.