What if you care more about unsafe pointer access than speed?
ASAN is great at detecting many unintentional memory errors, but it's not designed to thwart malicious attacks.
If you need a specific example of ASAN binary being exploited then see: http://int3pids.blogspot.ch/2015/04/confidence-2015-teaser-q...