If we talk in terms of concepts that exist in the C standard, we would say that you can't cast an object to pointer-to-X unless your pointer actually points to an X.

The reason your example is illegal is that you are casting to pointer-to-"struct derived", but the thing being pointed to is not actually a "struct derived."

The "physical subtyping" pattern works because the C standard says that a pointer to a struct, suitably converted, also points to its first member. So a pointer-to-Derived, converted to a pointer-to-Base, points at Derived's first member. But a pointer-to-Base doesn't point at a Derived unless that object actually is a Derived. So the downcast is only legal if the object actually is a Derived.