Sure, they can. But as said, they are still quite basic stuff compared to what IDEs do. It's better than nothing, but they don't contribute to productivity as IDEs do.
> Honestly, I think I just... haven't ever felt the need to do these things in Java?
This is surprising for me, but hey, each of us are a different use case.
My recommendation is to try an IDE extensively, and then you decide what suits you better. I did this, and never looked back (for Java programming; I'm still using Vim extensively for general text edition, for example).