However that still doesn't make it a valid argument to discard accessors entirely. Getters and setters play an important role in a critical principle of OOP, which is object encapsulation. They form the boundary between and object contract (or interface) and its implementation.
Every time you write code that directly accesses an object's field, you introduce a new dependency to how this object was implemented - and this is bad, because you should always depend on what it does, not how it does it.
And this is also true for plain "data objects" : they are objects, they provide data. You need to know what data they can provide; you shouldn't care about how they provide it. Leave that to the guy who write these objects' classes - and if it was you, learn to be schizophrenic.
Dependency to implementation just make maintenance operations harder, which IMHO is a lot more evil than losing a bit of performance. It's a trade-off I can accept easily, especially since computers and devices still get more powerful and most applications just don't require that much power to run smoothly - and if it does, you should optimize your code structure and algorithms instead of looking for micro-optimizations like that.