It's 100% fine to end the interview early as the OP did. The interviewers/hr will end the interview early the moment they're sure it won't work out.

The fact that the interviewers acted rudely is a great sign that it's an awful place to work. Would be nice if the company were named so others can avoid interviewing there.