I don't disagree, but couldn't consciousness play out at different time scales? If an alien lived for one second but its brain processed information billions of times faster than ours, would they be right to consider us non-conscious?
I think you are correct in this insight. I also think that the perceived [or relative] rhythm of the passage of time would have huge implications here in this case: from the fungal perspective, "our" world would be "seen" at very high speed, which would prevent any form of interaction. It would even prevent reciprocal detection.
This "perceived time rate" difference could happen on an astronomical scale and might contain the explanation of informational paradoxes such as Fermi's.