Hacker News Reader

The systems I've seen basically break things into different services. Tied together with gRPC or Thrift which have code generators for most languages. So the Java backend simply makes RPC requests to a server running R.

Although in one case we had very tight latency requirements (ie: 10ms) so the ML results were pre-computed and loaded from a cache on the backend servers.

View on HN

I imagine I will be breaking down into different services as well. An ML "blackbox" that makes a call to the back-end for data and returns a result/prediction. This could happen through an API. What kind of API to choose is still open.

I'm not very sure what you mean when you say the ML results were pre-computed?

If you're pulling data from somewhere automatically then you should make sure to define the data contracts well and that they're not changed. Also, A/B test things if possible. I've had issues in the past where the data pipeline view of the data and the API view of the data weren't the same. Or where a bug was fixed that resulted in the values of certain fields to change.

>I'm not very sure what you mean when you say the ML results were pre-computed?

We were scoring ads per page, and possible values for both were known ahead of time. So for each ad-page combination we generated the scores and then pushed them into a giant cache.

Reply on news.ycombinator.com