Comment by samwillis | Hacker News Reader

Comment by samwillis | Hacker News Reader

Parent Full thread

This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB (https://tanstack.com/db/latest/docs/guides/live-queries). We pass a RefProxy object into the where/select/join callbacks and use it to trace all the props and expressions that are performed. As others have noted you can't use js operators to perform actions, so we built a set of small functions that we could trace (eq, gt, not etc.). These callbacks are run once to trace the calls and build a IR of the query.

One thing we were surprisingly able to do is trace the js spread operation as that is a rare case of something you can intercept in JS.

Kenton, if you are reading this, could you add a series of fake operators (eq, gt, in etc) to provide the capability to trace and perform them remotely?

Yes, in principle, any sort of remote compute we want to support, we could accomplish by having a custom function you have to call for it. Then the calls can be captured into the record.

But also, apps can already do this themselves. Since the record/replay mechanism already intercepts any RPC calls, the server can simply provide a library of operations as part of its RPC API. And now the mapper callback can take advantage of those.

I think this is the approach I prefer: leave it up to servers to provide these ops if they want to. Don't extend the protocol with a built-in library of ops.

Ah, yes, obviously. This is all very cool!

Just a side note - reading https://tanstack.com/db/latest/docs/guides/live-queries#reus... I see:

    const isHighValueCustomer = (row: { user: User; order: Order }) => 
     row.user.active && row.order.amount > 1000

But if I'm understanding the docs correctly on this point, doesn't this have to be:

    const isHighValueCustomer = (row: { user: User; order: Order }) => 
      and(row.user.active, gt(row.order.amount, 1000))

Yep, that's an error in the docs..

As I understand it it's basically how Pytorch works. A clever trick but also super confusing because while it seems like normal code, as soon as you try and do something that you could totally do in normal code it doesn't work:

  let friendsWithPhotos = friendsPromise.map(friend => {
    return {friend, photo: friend.has_photo ? api.getUserPhoto(friend.id) : default_photo};
  }

Looks totally reasonable, but it's not going to work properly. You might not even realise until it's deployed.

Reply on news.ycombinator.com