You can absolutely use the word average to refer to the median when you are speaking in a non-academic context.

https://www.merriam-webster.com/dictionary/average https://en.wikipedia.org/wiki/Average