You're right but for the people who are confused, it's because of the semantics of the question. If you specify the question a little more and ask "What came first? The chicken or the chicken egg" then the question is how do you define a chicken egg. Is it the egg that comes from a chicken or the egg that a chicken hatches from.

In the first case, the chicken came first and in the second case, the egg came first. I think it's obvious the the second case makes more sense but some people might argue.