Right, but that's the thing: possession (even in the actual sense, let alone constructive possession) doesn't mean use. A card declaring (in effect) "I have been prescribed cannabis to use it as medicine" does mean use.

If anything, Grandma being the card-carrying pot user and not you would likely be evidence in your favor - to be presented and argued by a competent defense attorney in court, of course, not by you to some power-tripping cops interrogating you (the only correct response to a cop asking you anything for any reason at any time under any circumstance is for you to invoke your rights to silence and an attorney - and such a "shut the fuck up" strategy has indeed gotten people out of pot-related convictions, at least if these lawyers are to be believed: https://www.youtube.com/watch?v=sgWHrkDX35o).