andydunstall

From what I could see of FRP, it only runs a single server node so isn't suitable for production traffic (which needs to be fault tolerant, scale horizontally, support zero downtime deployments...)

Piko is also designed to be easier to host, so can be hosted behind a HTTP load balancer. That does mean Piko is currently limited to HTTP only, but that seemed a worthwhile tradeoff to make it easier to host

Few things:

- If your trying to access a customer network (such as for BYOC), exposing a public port in the customer network is likely a no-go (or would require complex networking to setup VPC peering etc)

- The Pico 'proxy' port doesn't need to be public (and in most cases won't be), such as you can only expose to clients in the same network (which is one of the benifits of self-hosting)

- The Pico 'upstream' port (that upstream services connect to) will usually need to be public, but that can use TLS and has JWT authentication

As commented below, Pico is already a well established name for a text editor so I've renamed to Piko: https://github.com/andydunstall/piko

Not yet (still quite a new project), its on the list to add one

Yep I checked out overlay networks, its definitely a very cool project. However it also seems pretty complex to host. I think they are different use cases

Could you elaborate? Do you mean tunnelling generally or this implementation?

I didn't know there was already a long-established project called Pico :)

As someone suggested below, I'll rename to 'Piko'

Not sure I follow

Pico is a reverse proxy, so the upstream services open outbound-only connections to Pico, then proxy clients send HTTP requests to Pico which are then routed to the upstream services

So as long as your browser can access Pico it should work like any other proxy

(Theres a getting started guide if that helps: https://github.com/andydunstall/pico/blob/main/docs/getting-...)

Good idea - will do that!

That demo only uses docker compose: https://github.com/andydunstall/pico/blob/main/docs/demo/doc...

Yeah sorry I started Pico before realising...

When Pico server nodes are replaced, the upstreams will automatically reconnect to a new node, then that node will propagate the new routing information to the other nodes in the cluster

So if you have a single upstream for an endpoint, when the upstream reconnects there may be a second where it isn't connected but will recover quickly (planning to add retries in the future to handle this more gracefully)

Similarly if a server node fails the upstream can reconnect

Very cool, I've been looking for an open source Ngrok alternative as well though for production traffic rather than development (I couldn't find a good option on awsome-tunneling so have been playing about with a proof of concept at https://github.com/andydunstall/pico)

yep - as i mentioned in a comment below - we arn't concerned about when the key is actually deleted - only that its not returned when looked up if its TTL is hit

sorry should have been more clear

>The integrity of a set of related keys requires that either all keys exist, or none exist

Sorry it wasent that clear that part of our fix was such that it no longer matters if all keys exist or none - we reordered the expires such that the invariants still hold even if all the keys dont still exist My understanding of expiry is - its not guaranteed when keys are expired if they are not accessed - but if a looked up keys TTL is hit it will not be returned - which is all we cared about

we reordered our keys such that they can expire at different times - also redis checks a keys TTL before returning - and wont return if its expired. i think it will 'eventually' delete if its not accessed, but we only cared about keys being returned, not when its actually deleted

sorry i should have explained. by ordered i meant after expiring a key its TTL will be set, which will never be a time before a previous expire. when it looks up the key, if its TTL has been reached it will not be returned (which we checked in the redis source). we weren't concerned about then the keys are actually removed

thanks for pointing that out - will take a look :)

yep good point :) - in the end it didnt matter much as we could maintain our invariants if we just reordered the expires

yep - part of our fix was ordering the expires such that the invariants are maintained even if the expires arn't exactly at the same time (we are assuming expires are at least ordered)

Hey, I'm the author of this blog post. Ask me anything!