There are Gemini clients that can inline images, so visitors to a site could decide to enable that if they wanted to see for instance a list of thumbnails.
There are also clients that make a second request to ask for a favicon. In the spirit of the protocol the "icon" is just a UTF-8 symbol, but that behaviour is still controversial.
1. gemini://geminiprotocol.net/docs/protocol-specification.gmi
2. gemini://geminiprotocol.net/docs/gemtext-specification.gmi
Expected? No, it's counter to the intentions of the community.
Tolerated? Maybe. Here's a fun one (saw this in one of the past discussions dang linked): https://github.com/makew0rld/amfora/issues/199. That was over favicons, but the response would be similar if linked images were automatically fetched. Though the protocol has a "backoff" return code that could be used to throttle those things that would be less disruptive than Drew's approach of banning specific clients.