Show HN: Lockbox: forward proxy for making third party API calls | Hacker News Reader

Show HN: Lockbox: forward proxy for making third party API calls

github.com

137 pointsjjak8231 comments

Just watch out, there's a whole class of vulnerabilities around tricking a more privileged piece of software into doing something on your behalf https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-de...

and that's not even getting into the assorted awesome HTTP request smuggling and SSRF vulns when 3 or more http implementations rub up against one another

I've usually used nginx for this purpose, proxying requests to a 3rd party server but tacking on authorization headers, for example, keeping them secret from the client. This strategy's always works great for me, but I guess it's limited to fairly simple pass-through use cases.

This seems cool if there are more complex needs or configuration that can't be done with simple proxying and nginx rules.

Nginx is quite extendable, there are tons of nginx plugins to help you add more customizations.

There is OpenResty, a version of nginx [0]. It allows you to script all sorts of stuff with Lua inside nginx itself.

Tools like lockbox are not necessary, nginx, caddy, etc or heck even a normal 70 line python3 fastapi based script works just fine and should be more extendable than lockbox.

[0](https://openresty.org/en/)

Tools like lockbox are not necessary, nginx, caddy, etc or heck even a normal 70 line python3 fastapi based script works just fine and should be more extendable than lockbox.

Well in that case you could make that argument about any piece of software.

There's always value in providing a simple tool for handling & automating the complexity found in 90% of use cases.

> Tools like lockbox are not necessary, nginx, caddy, etc or heck even a normal 70 line python3 fastapi based script works just fine...

A normal 170 lines of Python3 Flask based script, but who's counting?

https://github.com/mkjt2/lockbox/blob/main/lockbox/app.py

at that point its not worth it, to depend on an external provider, simpler to just code it myself and have 100% control, but you do you.

Btw cool startup! (ClearAlpha) , good luck with it !

I like the idea. It reminds me of the incredible issue with US banks where the most common way to setup 3rd party integrations with it is to give your username and password. Like, WTF?

It’s funny because as a passionate self-hoster, I consider as an advantage what you consider as a drawback.

and, to add insult to injury, it's my recollection that one must give those credentials to Plaid, not even entering them into your bank's website like a sane oauth2 dance. I always ensure I go to my bank's website and change my password afterward if I'm forced to use that MITM junk

I'm so jealous of PDS2 in the EU <https://www.berlin-group.org/> (although I guess it's easy for me to be jealous since I don't know if their banks actually implement the standards, but it's still a bunch better than the situation here)

They do, it's a regulatory requirement. Anybody who exposes an IBAN account, must have open banking apis exposed as well. So, all 5k+ banks have it.

US legislation with a 3-5 year mandatory adoption timeline would be incredible.

There's "implementations" that allows them to satisfy the requirement and then there are usable implementations. User experience is not great in some cases.

Do you know if the APIs are usable for regular people as well?

Not by regulation, you need a specific “payment service provider” licence for that.

I think you can apply to get access to some sandbox from your bank (at least one of my EU banks does this) where you get access to documentation and fake APIs. Then you can go through a lengthy process to have it approved, but in my case I think it also requires a proper business entity to move it to "production". This might differ from bank to bank.

Or, you can use something like GoCardless which provides a sort of aggregated API and "connection APIs" for making things simpler: https://gocardless.com/bank-account-data/

Not affiliated with them, but I do use GoCardless to automatically import transaction data from a couple of EU bank accounts to my local datastore.

I wish. It's only available to licensed AISPs that carry fun requirements such as:

> The license can be obtained within 6-9 months for the application fee of 6,800 EUR. The minimum capital requirement varies depending on the nature and scale of the AISP’s activities but is typically around 125,000 EUR. The amount is ultimately determined on a case-by-case basis by the Dutch Central Bank which is the regulator of the Dutch financial market.

I recently connected my Chase checking to my work expense system via Plaid, and it appeared to use an real integration - I was sent to a Chase domain to log on, which explained what data I was giving to Plaid _as well as_ what data I was giving to the third (fourth?) party.

I recall a similar experience linking Chase to my Schwab account.

I was pleasantly surprised that I was not asked to give my bank password to anyone but my bank.

Plaid does have OAUTH for some banks to make it more reliable.

Not a Plaid employee, just a dev completing their OAUTH approval form.

PSD2 is a nice idea but nightmarish in practice. I used to use an app to track my finances and it would often fail to properly establish a "link" after successful authentication with mysterious error messages, bank connections would just randomly fail to refresh their data for hours, even days at a time, their services provider (the party sitting between your bank and your budgeting app) SAID it would require re-auth every 90 days, but in practice it seemed to start requesting authentication after a month or so.

To access your own data, it needs to go from your bank, to a licensed Account Information Services Provider (AISP), to your budgeting app of choice, and finally to you. You can't utilize PSD2 without involving all of those 3rd parties because banks don't offer PSD2 APIs to consumers and I've only seen AISP services sold B2B. [1]

I'm someone who has no issues using 2FA everywhere and carrying around a Yubikey, but even I got fed-up with constant re-authentication requirements: Enter bank email & password, next, next, confirm, approve, yes I'm sure, now take out your phone, unlock it, open the bank app, log in there, confirm that prompt, yes, i'm really sure. Then it takes 1-2 minutes to "connect" your account and load the latest data. Repeat for every bank account, every 30 days.

It beats trusting your banking password to some random 3rd party, but it's a pretty terrible experience as an end-user if you ask me.

[1] Someone pointed out GoCardless which seems to accept private customers as well, a welcome surprise.

Why are banks so resistant to this?

This is super neat! I like the idea of better auditing of _what_ data gets sent over these integrations, not just what integrations are currently set up.

We do something similar with our tool (generative code thingy), where users can make API calls to various services (without credentials), and then we add the necessary auth tokens/headers for their code to actually run.

I'd love to give folks better control over their own data/credentials, Lockbox could be an interesting workaround without us having to build anything. Thanks and nice work!

Great to hear this might be useful to you!

Also from Stripe team (used in past, and believe still running without issues to this day)

https://github.com/stripe/smokescreen

Smokescreen is a HTTP CONNECT proxy. It proxies most traffic from Stripe to the external world (e.g., webhooks).

Smokescreen restricts which URLs it connects to:

It uses a pre-configured hostname ACL to only allow requests addressed to certain allow-listed hostnames, to ensure that no malicious code is attempting to make requests to unexpected services. It also resolves each domain name that is requested, and ensures that it is a publicly routable IP address and not an internal IP address. This prevents a class of attacks where, for instance, our own webhooks infrastructure is used to scan Stripe’s internal network. Smokescreen can also be further configured to allow or deny specific IP addresses or ranges.

Maybe the most amazing and ultimate end-game here would be an API trust layer that can separate the most private data from 3p APIs.

- Location anonymization to fetch any resources like restaurants.

- Sensitive personal data encryption to verify credit info securely.

My guess is not all of the above could be easy/possible, its a leaky abstraction to think of APIs that way. But the biggest bang for the buck could be for privacy in the most mundane of usecases.

I like the core idea, but I fail to see why this is a good solution. IIUC the workflow is:

1. Run a Lockbox server

2. Define the API request in automation tool as a webhook.

3. Proxy it via lockbox to add auth.

All this really does it allow you to revoke auth at any time. As they can freely use the key. I see there are plans to restrict the requests allowed but doing that on the fully composed request it going to be extremely difficult to do reliably.

If you are already hosting some API why not do something like this instead.

1. Run a Lockbox server.

2. Send a webhook from the automation tool that just contains the relevant variables.

3. Lockbox composes the variabless into a request.

Basically if you want any sort of security and access control that is reliable you just want to send the parameters of the operation to Lockbox, then have the actual request generated on your server. Rather then doing the request generation in the automation tool then trying to validate that it was generated correctly.

Do IFTTT/Zapier even allow switching the Host for such requests? For companies like Zapier, it makes sense to request complete access to Google Sheets, but it gets very problematic for least-privileges, and this is a reasonable solution. I've even considered building something similar in the past, but getting vendors to switch hosts is not always easy.

That's nice. I was wondering, is it possible to tell Zapier or other tools to go through a proxy or use a different API URL for the services you configure there? E.g. tell them to use the Stripe API but call it at https://foo.mydomain.com instead of api.stripe.com?

This is a cool API but I wonder if something like Zapier would support redirecting their slack integration via your onsite proxy.

> You can restrict access to external APIs in a more fine grained manner

I've done this very successfully for many APIs before. I've found weird things among providers where using AWS (or GCP or Azure) gives you crazy fine-grained access controls (which are great after you spend two days figuring out how to use them well), but, some of the low cost competitors have core services that work just as well but their APIs are entirely binary (either you have full access, or none) but by adding a thin API layer above that you can enable all sorts of useful things. Provider doesn't support R/O API keys for terraform plans? Just run a proxy which enables that. Provider won't allow you to give someone an API key which can only reboot VMs, but, not delete them? Just write a proxy which does.

Very powerful model if you adopt it well.

(Zapier co-founder) Perhaps the least known feature of Zapier is that you can use the dev platform (https://zapier.com/developer) to make your own private apps for any API (including internal/private services) and then sign requests with auth credentials. The dev platform even has a full WYSIWYG app builder! Then, anyone in your account can use the private app to make Zaps.

Relevant to OP, all platform auth creds are secured and not exposed after initial setup.

Separately, we just launched custom actions which allows anyone to extend existing public apps on Zapier with new endpoints (https://help.zapier.com/hc/en-us/articles/16277139110157-Cre...)

This looks super interesting - gonna check it out.

Noho ora mai

Kapai bro

Thank you for taking the time to write such a clear and succinct readme.

I really liked the layout:

  - Why
  - How Lockbox helps
  - Main benefits
  - Drawbacks
  - How to run
  - Design philosophy

I wish more projects would follow this format :)

Reply on news.ycombinator.com