I watched a series of college lectures on the history of language a couple of years ago, and that was one of the main points the lecturer made. Growing up in a traditional school system, I had never really thought about it that way.
Having said that, there is also a "high" and "low" version of most languages. The "high" version is full of obtuse rules and associated behaviors, while the "low" version is more catch-as-catch can.
So yes, you are correct: enjoy the art of your native tongue! I love screwing around with the rules. But also be aware that the culture you are in may prescribe all sorts of silly little rules, and knowing these -- and when to apply them -- can drastically affect other people's perception of your character. Whether we like that fact or not.
One of the things I wish I had more time for is the study of languages and the philosophy of languages. It has direct application in everything from AI to talking to Dolphins to programming projects. Really cool stuff.
"I passed the man in the red hat by." pass-by
"I threw all of my old, useless papers out." throw-out
To repeat the parent's point: the stricture against ending a sentence with a preposition is simply a bias for grammatical structures that English inherited from Latin (through French) and against structures inherited from German.
The rule against splitting infinitives falls into the same category. It is not possible to split an infinitive in Latin because infinitives aren't made up to two words.
Again, these two rules (and others like them) are about biases, not about grammatical correctness or even clarity of communication. When you say "Who am I speaking with?" and "To boldly go where no man has gone before.", no one is actually left scratching their heads and thinking "I wonder what he means?"
"I passed by the man in the red hat."
"I threw out all of my old, useless papers."
Both of those not only keep the preposition away from the end of the sentence and keep it near the verb to which it belongs, but most importantly they sound more natural.
Your example of "Who am I speaking with?" follows this same pattern -- "with" is kept next to the verb "speaking".
If "With whom am I speaking?" sounds snooty, then I guess I am a snob.