It depends on what the struct contains. I have developed many Go API's professionally at several companies since Go 1.1, and all my servers and up looking like a server struct with only a few fields - a database, AWS client object, and some prometheus metrics. The logic is typically split among many files, all implementing receivers on that struct.
If you have independent, different elements in that API, you break them out into separate "servers" but still register the endpoints on the same HTTP handler.
I know that people don't like external libraries too much, but I'd like to plug my own here. You declare your API in OpenAPI 3.0 (aka, Swagger) and it generates your server and models for you, so all you need to do is write the business logic. (https://github.com/deepmap/oapi-codegen)