> As a millenial with hope of some sort of real estate in my future, I've been thinking of buying an empty/distressed lot and rebuilding it, but I don't know how realistic that is.

Very reasonable. I've done this before. Let me know if you'd like to chat over email, and I'll get you my contact info.