The whole abstract sounds like marketing copy from those websites that don't tell you what their product is:

> By avoiding high depth, shortcuts, and intricate operations like self-attention, VanillaNet is refreshingly concise yet remarkably powerful.