It's absolutely true that you can use ORM and scaffolding patterns in a totally secure way. But the problem is that the defaults are insecure -- every table can be accessed, every record is available, every field can be edited, and the URLs for doing so are (deliberately) easily guessable.
One of the simplest and most fundamental rules of effective security is to close everything down by default and only open things up as required, after careful consideration. Scaffolding breaks that rule.