Well yes, you shouldn't pay tax on that, and it would cause huge problems. If your house increases in theoretical value should you pay tax on the increase? Or only when you sell the house?
You do understand this is exactly how property taxes work, right?