It does and I understand. But in my experience most of what should be tested is not "does the database contain x?" but "does this where clause filter y?"

Those sorts of things are programmed in EF by LINQ queries. All I'm saying is, test the LINQ queries the same way you would test non-EF LINQ queries. LINQ doesn't require a database.