You can have neurons connected to each other in ways that resemble the geometry of higher dimensions. Like, imagine that the vertices in this image are neurons, with the edges being connections between them:
https://upload.wikimedia.org/wikipedia/commons/thumb/4/45/Di...