We track the readset for any active subscription. When a new write transaction commits we compare the writeset for the transaction against any active subscriptions. If there's an intersection then it's likely that query will have been invalidated, so we rerun it.

I talk more about how we perform this comparison in this (now rather outdated) talk: https://youtu.be/iizcidmSwJ4?t=1218

Note that there are false-positives here, i.e., it's possible for the inputs to a query function to change without the outputs actually changing, but this is not a significant issue in practice.