Anything with a probability mass or distribution function has entropy; the state space of Markov chains just tends to get combinatorially big as the chain's memory increases.