There has been an instance of a Governor General dismissing a Prime minister.

https://en.wikipedia.org/wiki/1975_Australian_constitutional...