Here's the odd thing - if I meet a candidate who
knows about shellsort, I'd already be impressed. Many people I interview have trouble with much more basic concepts than that.
And of course, it is worth keeping in mind that when you hire an employee that hopefully at some point, the load demands are higher. And it's kind of awkward if your product hits rapid growth, but your engineers are currently busy learning about algorithmic efficiency.
Nobody sane asks about "worst-case efficiency of shell sort" unless you yourself suggest shell sort as an algorithm. But it is expected that you get the general idea of O notation, because it actually does matter for day-to-day work.
And, as a practical interview tip: It's perfectly OK to say "I don't know about this specific case" - just follow it up with "for most sorts, O(n^2) is an upper bound, so if I'd guess, I'd go with O(n^2).", or something like that. It tells the interviewer that you know general principles.
That's the big problem in most interviews - candidates treat it like a test, where there's just the right answer, and nothing else. It's not a test. It's an attempt to somehow, within an hour usually, find out what you know and how you think. Throw the interviewer a bone or two, and help them do that. Silence and short "I don't know" answers really just waste time for everybody.