A few ways.

rm ./~ is likely the easiest.

Another option, shell dependent, would be to turn off shell globbing.

The `GNU` version of `find` has a `-maxdepth` option so

find . -iname '~' -maxdepth 1 -exec rm {} \;

would work, but I don't like relying on `GNU` extensions.