They are arrows. Arrows and objects are the primitives of category theory, just like sets and set membership are the primitives of set theories (well most of them anyway).
A category that captures the notion of sets and maps can be described in terms of arrows, objects and some semantic rules about which arrows (here interpreted as maps) connect which objects (here interpreted as sets).
Wiki on the "Category of sets [and maps]" -> http://en.wikipedia.org/wiki/Category_of_sets
Other rules about arrows between objects describe categories that capture other mathematical concepts.