Loom, I think, was like that sometimes, but Monkey Island wasn't too bad. Sierra's King's Quest series, however, was notorious in this respect, where the thing to do was something arbitrary you could not reason to in a million years.
That's the kind of thing that sucks in adventure games. In real life, I can be creative to a very large degree because reality has content, specifically, to the degree that I grasp that reality. A ball of clay is a ball of clay, and because I understand what clay is, I know what I can do with it. But in a game, a ball of clay is just a symbol. There is no ball of clay. It is a nominal entity, an image or word plus some code that simulates maybe some aspects of it in very constrained ways (in adventure games, this is truly very narrow, as in, if user clicks these two things with this action, then show picture and display text, the end). And this might be okay if the author has arranged things sensibly, but sometimes you end up with truly bizarre ideas. You wonder how these things passed QA.