It may not be realistic, but it is conceivable.

The Road Not Taken by Harry Turtledove is a fun take on something like this concept.

Couldn't you just launch a sufficiently fast and heavy object into another civ's planets?

I don't see how nuclear weapons are necessary for destruction if that is your point.

Maybe if a civilization evolved on a world extremely poor in heavy metals, they'd never get around to fission. Maybe fusion bombs will forever be impractical without a fission kickstart?

If there's some obvious-but-not-to-us way to break Newtonian physics without ever figuring out atoms... maybe.

I civilization of intelligent funghi?