Yes, the penny should be gone.

But I disagree about the $1 note. According to the GAO, replacing the $1 note with a $1 coin would actually result in a net loss to the government:

https://www.gao.gov/assets/700/697778.pdf