Plain text is ok for the README file, but markdown is more readable, specially if you are going to include code examples.
Visually, having a decent README sets the project apart, and makes it look more "put together." Anecdotally, I can say that when I'm looking for a library to solve a particular problem, I'm probably going to stick around and read more about the repo that has beautiful markdown documentation, all else being equal. Maybe I'm just vain, though... :)