The Orbit of the Moon around the Sun is Convex

The Moon is slowly moving away due to tidal interaction. If the Earth and Moon were tidally locked (both presenting the same face to each other at all times) then the orbit would remain stable. If the Earth would be spinning slower than the Moon, then the Moon would be slowly pushed inward.

An interesting consequence is that in a billion year's time, after Earth's oceans would have evaporated, this tidal interaction would greatly reduce, "fixing" the Earth-Moon system in place.

More here: http://www.talkorigins.org/faqs/moonrec.html

Ellipses are stable orbits too.

As caf said, and as Johannes Kepler first correctly deduced, stable periodic orbits in a two-body gravitationally bound system are ellipses with an eccentricity (deviation from a perfect circle) dependent on the system. A perfectly circular orbit (eccentricity 0) is of course theoretically possible, but would require careful fine-tuning. Earth's orbit is rather close to a circle, as is Venus's, but Mercury, Mars, and especially Pluto have notably eccentric orbits.

The case of a system of more than two bodies is of course trickier. One cannot analytically solve the orbits even in a three-body system, not to mention a real planetary system. The orbits are chaotic and everything perturbs everything else in complex ways. This does not mean that the orbits are intrinsically unstable or that their evolution cannot be numerically predicted, however. For reasons presented below, for most practical purposes the orbit of any planet in the Solar System can be approximated as an ellipse.

There's a selection effect present: in areas of the protoplanetary disc where a nearby growing planet caused major disturbations, another planet couldn't form in the first place, and those planetesimals that did form, would eventually collide and merge with their more massive neighbor, or in extreme cases, be ejected from the Solar System altogether. This is the "cleaning the neighborhood" effect that is now used in the definition of a planet.

Also, if two planets have orbital periods close to a simple integer ratio, such as 3:2, the resonance magnifies their mutual perturbations over time. Unsurprisingly, there are no such simple ratios in the Solar System.

Every contemporary planetary body has thus been selected for having an orbit stable over a timescale of at least billions of years. The asteroid belt consists of remnants of the original disc: a fifth rocky planet could have formed there if not for the effect of Jupiter, preventing the coalescence of anything larger than Ceres. The belt was also originally much denser than currently; over the eons Jupiter has pruned every rock unfortunate enough to have an orbit just a bit too unstable.

Based on observations of other planetary systems, we now know that a giant planet could form or otherwise end up in a very eccentric orbit and thus prevent other stable orbits in a large part of its planetary system. This was surprising, given the nearly-circular orbits of the giants in our system. In retrospect, it is clear why we observe our system to be so well-behaved - otherwise, Earth wouldn't have formed and we would not be here making these observations in the first place! This is an instance of the so-called Weak Anthropic Principle.