As it turns out, some words have connotations which are bad, and by using them you're communicating something that isn't an example of what you want to reinforce in a child, or anyone at all really.
So I am not suggesting that GP should be ostracised over saying 'retarded' but I think it's important to atleast question why they are using language which hurts a specific group of people that have done nothing to deserve it.
Here's a quote from a website[0] that explains why using 'retarded' in the way GP did here is harmful:
“It hurts and scares me when I am the only person with intellectual disabilities on the bus and young people start making “retard” jokes or references. Please put yourself on that bus and fill the bus with people who are different from you. Imagine that they start making jokes using a term that describes you. It hurts and it is scary.” – John Franklin Stephens, Special Olympics Virginia athlete and Global Messenger
Hopefully that quote helps to see why no one would question the usage of 'silly'.
[1]: https://www.spreadtheword.global/resource-archive/r-word-eff...