I would say that if we presume memories of everyone involved have been modified, that is an equally strong predictor that we are in a simulation.
But what if we discover that our universe is made from tiny voxels or something like that, that will be undeniable evidence, right? Wrong! Who says that real universes are not made of tiny voxels? It could be [1] the other way around, maybe real universes are discrete but their universe simulations are continuous, in which case the lack of tiny voxels in our universe would be the smoking gun evidence for being in a simulation.
[1] This is meant as an example, I have no idea if one can actually come up with a discrete universe that admits continuous simulations, which probably should also be efficient in some sense.