Well, that's simply not how it works. Sure, pretty much everyone has lunch in the cafes. Breakfast is popular too but you can get it until 9:30 or 10:00, so you don't even have to show up early. Dinner attendance is significantly lower, and pretty much everyone I know who regularly has dinner at work goes home right afterwards.

Personally I probably manage to get breakfast once a week on average and stay late enough for dinner 2 or 3 times a year.