Does that track with your understanding?
A buy market order would try to match with the "best price" which in a deeply crossed book would mean matching with a really low priced sell order. Exchanges match orders in price-time priority. Similar is true for a market sell order - would match at an extreme high price.
Besides the midpoint of the order book, another metric for a "current price of the stock" people use, is the "last trade price". In the situation above you would get "swings" in the price because market orders would be trading very high and very low if they alternate between buying and selling. The data structure on the exchange itself isn't "swinging", it's just the overlapping region being slowly eroded by market orders. The "last trade price" metric looks really insane in this situation.