This thinking is flawed in two ways. The glaring flaw is in the restrictive definition of patterns. The more subtle flaw is in not recognizing that they have patterns of their own at similar scales to the ones that were rejected. Abstraction is extremely powerful, but operating at higher levels of abstraction doesn't appear to imply higher productivity or reduced needs for patterns as a medium for sharing context."
The reason we don't get a boost in productivity with higher abstraction is because (a) we have to build such abstractions out of a great number of smaller pieces, so that 'greater abstraction' means 'more work', and (b) it is just very difficult to map our mental models into reusable abstractions. Not very many people can do it well.
Yes, every language has 'patterns', but we call those linguistic structure, syntax, or grammar. "Design patterns" refers to something else, (or if it doesn't, it is just a confusing domain-specific renaming of the above.) The problem with design patterns is that they are ostensibly solved problems, and thus people shouldn't be implementing them except pedagogically.
Take arrays for instance. There is no "Array Design Pattern", and we don't ask people to implement arrays every time they want to employ them. We ask them to use existing array structure. Yes, you should understand how they work. You probably should implement them once or twice and know how to recognize them. But you do not say "Oh, just use the array pattern here. Start by rolling your own array, since our language doesn't have array syntax."
We build arrays into the language so that they are easy to recognize and the pattern becomes a primitive tool. If you have to keep rolling your own, you'll never get past that. It will never become primitive, and you will stagnate. That is what those who object to design patterns object to: by thinking of them as elements of design -- the work of day-to-day programmers -- we won't provide linguistic abstractions that allow them to become primitive tools. We'll be stuck re-rolling our own arrays because people think that this is what programming is all about. Tedium.