Concrete is more difficult, but there's still a trick to it. The blocks you use for breaking have a bit of flex to them, but are also extremely brittle. So just getting it to flex just a small amount will cause them to break in half. That's also why you place spacers between stacks of bricks or wood. You're leveraging the flex of the material to break it, and stacking them without spacers makes too rigid to break.
In short, this stuff is 90% magic trick.