A good place to start might be the “Evaluation” chapter of Manning et al.‘a “Introduction to Information Retrieval”: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation...

The whole book is superb, and while some of the content is a bit long in the tooth at this point, the evaluation chapter has aged extremely well.

Oh, and if you’re interested in how to evaluate search engine user interfaces, the equivalent chapter in Hearst’s book on Search UI has you covered: https://searchuserinterfaces.com/book/sui_ch2_evaluation.htm...