I also simply choose to eat out a maximum of 1 or 2 times a month. I can afford 20GBP, I just have better options.
In fact their time can be far more valuable (into what money they can command by doing some extra work), so that the opportunity cost between cooking and buying lunch to make it not worth to cook.
So I guess restaurants that sells food are having a hard time, but restaurants that sells a service have a different story to tell.
[1] See how in the first case it's "to eat in a restaurant" and in the second it's "to go to a restaurant". To me that's the big difference : when you are here to fulfill a natural need (eating), you don't want to pay a lot for it, while you are here to have a nice time out, you will gladly pay more.