Rather than simply embedding the query, you may want to look into query expansion techniques. You might find that expanding the query semantically and then using more old school (read: extremely fast) tools like bm25 or tf-idf are pretty good already.

Your fundamental problem is that short snippets don’t have a lot of semantic meaning, but long snippets lose specificity. So one way of dealing with that would be to include staggered snippets, like bricks in a wall. This is ultimately a hack though, and for a lot of queries it doesn’t even make sense. If it’s a semantic match and not a direct text match, what does it even mean to give people a short segment?