There are many different cultures in the US. "you should come to my house!" is an expression in some parts - an expression that has gotten trouble for people in other parts who take the expression literally and show up when they were not wanted.
In other parts (the south, but I'm not sure where, maybe North Carolina?) it is an expression that just means nice to see you.
Note because it is an expression in some parts you hear it often from those people. People from the other parts don't invite people over often so they rarely say it. Thus the impression that it is universally used, when it is only a small region that says it often.