Ask HN: To store display results in a search index or not to
The search is driven by an Algolia-like service.
At the moment, we storing both the attributes that are used to search and the attributes that are used to display data. Most of the time, they are the same.
This design is great because it made the initial rollout simple.
However, there are a few problems that we already identified:
* There is no way to display sensitive data without an additional roundtrip to the server. We cannot store it in the search index, and if we are doing the roundtrip anyway, then it defeats the argument that this way is faster. * The entire index needs to be resynced every time a new attribute is added to the display dataset. * Engineers need to know how to update index data in addition to knowing how to API. * Potential for stale data in cards when queue times are long
These are a few arguments that make me want to switch to a different design, where we only store attributes used for search, and the only result we return is the search position and ID of the user, which is then used to fetch data about individual search results from the API.
I cannot see many downsides to the new approach, short of the additional roundtrip.
Is there anything else I should consider?