Show HN: Burrow is a globally distributed, serverless HTTP proxy | Hacker News Reader

Show HN: Burrow is a globally distributed, serverless HTTP proxy

github.com

76 pointsmyzie11 comments

The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal? It also seems problematic for re-use since not all accounts have those regions enabled. Perhaps using something like the below to discover enabled regions at deploy time would make it more "consumable" by other folks:

  aws account list-regions \
    --region-opt-status-contains "ENABLED" "ENABLED_BY_DEFAULT" \
    --no-paginate \
    --query "Regions[].RegionName" \
    --output text

That's the approach I've taken with SAM/Cloudformation deployments[1].

[1] https://github.com/mlhpdx/email-origin/blob/main/scripts/dep...

Aha, indeed. A few interesting notes on this. And thanks for the link to your Makefile.

I see you're a Cloudformation Stacks user (good stuff), so you may know some of what I say below already, but maybe the Terraform specific notes will be of interest.

Many AWS deployments (Terraform or otherwise) are single region unfortunately, with a primary factor there being that AWS APIs are for the most part region-specific. The AWS Terraform provider was built with this mindset too, unfortunately. Then the situation is complicated by Terraform's lack of support for dynamic providers, which is one of the most discussed still-open issues in Terraform.

Some related open issues:

https://github.com/hashicorp/terraform/issues/24476 https://github.com/hashicorp/terraform/issues/25244 https://github.com/hashicorp/terraform/issues/19932

My understanding is that people that have this as a first-class concern use something like Terragrunt to dynamically generate the HCL as needed.

https://terragrunt.gruntwork.io/

For Burrow here, I was was focused on shipping something and didn't want to spend time on generating that code just yet. But the project has gotten some attention so if this is a recurring request I'll probably do it.

Note that I intentionally chose the 17 AWS regions that are default enabled in every new AWS account, so it should only be if you want to customize it that this becomes important.

A Cloudformation Stacks based deployment to achieve multi-region would be another good (and perhaps better) option here. I could add that to the Makefile, with inspiration from your link there.

Oh I realized I was thinking of Cloudformation Stacksets, since those like you automate deployment of a Stack across multiple regions.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

Yea, unfortunately dynamic provider blocks to use a loop to handle this type of thing have been on the wishlist forever. IIRC Hashicorp's response is that it simply isn't possible, but I believe the OpenTofu project was targeting this (or may have already done it).

I manage a particular file right now that has 100 entries of something very similar. Just silly and annoying to search through and prone to copy/paste errors (one time I accidentally deployed something to a prod account in this way).

> The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal?

Yes, we do not have dynamic providers yet.

I lost patience and switched to CDKTF and then Pulumi soon after.

Very cool. Is there a reason to choose AWS lambda over other tech (say Cloudflare workers)?

Cloudflare Workers are great in general. The way they route client requests to the nearest Cloudflare data center means that a single client will most often end up with a small number (perhaps even 1) of public IP addresses coming out of Cloudflare. And to my knowledge, there's not a way to ask for your client to be routed in a different way. So due to this, AWS Lambda's inherent region-specific behavior is actually an advantage for Burrow, since we can ask to be routed through a specific region (no matter how distant) by virtue of hitting the per-region Lambda URLs.

That said, you could actually layer the two together if you were so inclined, and/or optionally route through Cloudflare and AWS.

Making the Cloudflare Worker implementation of this would be trivial and there's no dependency on Go in any way for the proxy itself.

Indeed. As another example, I deploy regional API Gateways for "same region" access to service stacks and then tie them into the "global" service using Route53 latency based routing records (one per region). It works nicely to get the best of both worlds.

Nice. I was wondering if r53 latency-based routing would work as an optional add-on to this. I didn't look into it but was thinking it might not work because the latency based routing records are A records that have to point at a fixed IP address right (rather than the function url in my case). But presumably API gateway just gives you hostnames as well so maybe I'm mistaken here?

With API Gateway the resource has a formulaic domain name, but in Route53 that is hidden because you're creating an "alias" record. I'm not sure the same thing would apply to Lambda URLs.

It's not exactly what I'm talking about, which is unfortunately private source, but here's a close public example of something similar (MX records rather than A records). This template get's instantiated in each supported region (a small set, because SES email receiving is only supported in a handful of places):

https://github.com/mlhpdx/email-delivery/blob/master/templat...

Cool idea. What was your initial use case for creating this?

Also, are the Function URLs publicly accessible? If so, what safe guards are there to prevent abuse?

Reply on news.ycombinator.com