If there are separate sections in the office documents that you can pull out and index as separate fields then you should do that. For example, if you were indexing patents, you would want to index abstracts and claims into separate fields.
If there are separate sections in the office documents that you can pull out and index as separate fields then you should do that. For example, if you were indexing patents, you would want to index abstracts and claims into separate fields.
At my last job, we used to index large documents on Solr. The largest chunks of them were indexed in non-stored fields. Which means they were searchable but you couldn't retrieve the actual text. This drastically cut down on the index size and the resources we needed to support it.
Then after scoring you'd return the top hits to the main application as IDs, which could retrieve everything needed (full text included) along with all the other information they'd retrieve for the document and generate the actual output.
What is the realistic expectation here. Should we say 50 mb. How everybody else do?