Both of those are typically referred to as manifolds with boundary. The setting of Stokes Theorem is often manifolds with boundary.
It depends on which mathematicians! Plenty of differential geometers allow manifolds to have boundary, and say "closed manifold" (https://en.wikipedia.org/wiki/Closed_manifold) to emphasise when they are dealing with a (compact) manifold without boundary (or, as you point out, really a manifold whose boundary is empty).