That's fine, but generally people on use that expression in a negative way, so I think it's reasonable to ask for explanation of why you think that.

In general I don't think it's fair to leave drive by comments like that without elaboration, as others might just see it and take it at face value when further clarification might reveal a misunderstanding.