I'm trying to think of other modifier examples. Not a perfect analogy but I don't think you'd say that, in English, the <h> in <sh> is silent.

I wonder about Italian, though. Would it be correct to say the <g> is silent in <gli>?

Seems a much tougher call in Italian. I would say /d͡ʒ/ has a much closer relationship to /dʒ/ than /ʎ/ does to /gli/ for example.