Show HN: File0 – An easier way to manage files in serverless apps
file0.dev
I can't be asked, so I built FILE0. It's for storing files but you don't need to complete an online course.
file0.dev
I can't be asked, so I built FILE0. It's for storing files but you don't need to complete an online course.
I've come to think the UX requirements of enterprise and indie dev / start up customers are fundamentally incompatible in the cloud.
As an enterprise I want to be able to set restrictions at the org, project, team and resource level for various compliance rules. And I want these rules to be restrictive by default (e.g block public access).
However as an individual developer the mere existence of those options makes getting started and debugging things painful. It's why I'll never again use Azure for a side project, although it'd be my first choice for a government project.
The nice thing about this service is that it seems to be public first. Which reduces the user effort involved in managing a permissions layer on top of storage.
Consumer cameras remind me of this when they started mimicking more and more sophisticated feature sets of professional cameras. Sure I want that fantastic sensor, great lens setup and ability to take both narrow and wide depth of field photos but I don’t know or rather want to be bothered dealing with adjusting all my f-stops, focal point etc. I need automation where possible for these cases to give me sane results.
In that case it’s a bit more complex because you have engineering problems for those intermediary steps (e.g. autofocus and making sure the image results meet criteria) whereas for software profiles you can often get away with just choosing option sets. Some options still can’t be hand waived away though and require some degree of looking at other things to inform them or being manually set even for pure software controls.
Once you get your own little pulumi/terraform library, these things really stop to be a pain.
My company is what people would colloquially refer to as a startup, although it is by most formal definitions an established company (more than five years old, profitable and self-funded, no external money, defined product and stable customer base).
Under the US SBA classification we are a small business by revenue, and we have three people.
Little frustrates us more than sophisticated features being locked behind enterprise packages, because we tend to select toward enterprise feature sets by default. We are small, but we make it a priority to do business in ways that most small businesses don’t focus on. This has been wonderfully successful for us because we have the kind of customers who will ask “how do you manage backups, and why should I trust you to do that?”
We can answer those sorts of questions with the same kind of robust architecture people expect from much larger providers. We can point to adhering to the 3-2-1 rule with three different copies of customer data (one production, two backups) maintained in three locations managed by two (legally distinct) clouds/data centers, each at least 500 miles apart, two of which are resistant to ransomware attacks (the backups in S3), all of which are protected with hardware MFA. We have a fourth copy as a failsafe in the form of rolling VM backups made every 24 hours, saved for approximately 7 days.
That is in large part due to using S3.
Sophisticated feature sets are extremely valuable for us even if we aren’t an enterprise. They allow us to put our money where our mouth is.
SaaS vendors appear not to have received this message, however. SSO is often only available as part of “Enterprise” pricing, which assumes either a huge number of users (minimum seat count) or is force-bundled with other “Enterprise” features which may have no value to the company using the software.
If companies claim to “take your security seriously”, then SSO should be available as a feature that is either:
- part of the core product, or
- an optional paid extra for a reasonable delta, or
- attached to a price tier, but with a reasonably small gap between the non-SSO tier and SSO tiers.
We have found that a lot of the thinking behind locking feature flags behind enterprise pricing is that there’s a perception that providing those features always comes with an increased support load. Or that you only need these features if you have lots of money to spend anyway. Neither have proven true for us.
Sometimes enterprise pricing is to offset the costs of and somewhat conceal that lack of focus on those features. It’s exceedingly ridiculous in 2024, for example, to have to email a support contact the SAML certificate to setup SSO. (In our case, we run away from those kinds of providers anyway.)
In direct reply to u/contrast: Of course there are some areas where we purchase the enterprise option because it’s the only thing available (our ERP for example), but that’s becoming rarer than it used to be. Where it becomes a deal breaker we usually find that the competition is happy to have us. Alternatively we make our own solution variously on platform agnostic primitives like S3 (or S3 API-compatible options), as a custom app in our ERP, or by using (and/or sponsoring) FOSS upstreams for commercialized source-available products. Being a customer that typically doesn’t need to talk to sales or support seems to make us a more profitable customer, and there can sometimes be room to negotiate there.
Edit to add: We don’t necessarily position ourselves as an enterprise grade provider. We tend to avoid engagements like that purposefully. Rather, we position ourselves as a trustworthy provider that takes their work seriously. We don’t find enterprise branding particularly helpful, and we aren’t oriented toward a sales culture or pushing to grow the business every single quarter. We prefer to simply do a good job and earn the trust our customers place in us. That does mean we need to operate with an enterprise grade focus in some areas, but that doesn’t mean we can or want to pay enterprise grade prices for every single thing we need.
We target mainly small businesses. Many of our customers want something different than what is frequently not-even-bargain-basement service that they had before. For example, we manage many customer domain names. Many of our customers have been burned in the past by web designer sole-props saying “yes” to any business that comes at them, but forgetting or not knowing to do things like annual WHOIS contact reviews, properly offboarding resold accounts, not implementing strong MFA, staying on top things like the recent DMARC changes, etc. These businesses deserve top notch service just as much as an enterprise, so we strive to do that for them. Unfortunately rendering that service frequently requires tools or features presumed to be desired or needed only by large enterprises.
I'm not sure how to use this from my Java or Rust projects. I also don't see any API docs so I don't know how to write a wrapper. I guess it's a project for and by Javascript developers.
I can't find anything about egress limits, file size limits, how incomplete transfers are handled, in what country the data is stored, and what happens when you try to create a file that already exists. Maybe that info is behind the login wall?
At the moment the client package is only for js/ts devs. The package is based on HTTP api calls so it shouldn't be a huge issue porting it to other languages, but obv it's challenging without public HTTP specs. If you're into implementing a wrapper I'd be happy to assist and share those details.
About costs. The is no information, because there is no egress fee. FILE0 is built on top of R2. They don't charge for egress, so neither you pay. File size limit is 5GB soft limit. This is set as a sensible default but can be increased on demand.
The main location is in us-east, and it's replicated around the globe. If the file already exists, it is overriden. Yes, all the setup guide and API-specific tutorials are visible in your dashboard after signup.
The first step is to get to a scale where you piss off the Cloudflare sales team. FILE0 is far from that. Whenever that will be the case we can think about solutions, but this wouldn't be a good enough reason not to use them, and the free egress until we can.
So any one of my users could overwrite or delete my other users files? Seems like this is not really thought through.
If the point is just a "everyone can do everything" bucket then that isn't too hard on any of the current blob storage providers.
In the frontend you need get a file-scoped token from the server.
Server: import { f0 } from ‘file0’; const token= await f0.createToken(‘myfile.png’);
You can send this token to the client. And use it like this: import { f0 } from ‘file0’;
await f0.useToken(token).set(myfileblob);
The docs are in the dashboard only after account creation atm. Public docs on the way.
What you are saying is that for actual usage I would need to
---
1. Read the docs for what access you provide by default (anonymous access, etc)
2. Build a backend api endpoint to do all AuthN/AuthZ checks, call your library to generate a token and then return that
3. (On the frontend) Make an API request to my backend, get the token. Call your library with the token to upload the file
4. (maybe think about revoking that token to disallow overwriting the file with the same token)
5. In other clients use your library to retrieve the file? Do I need to build a backend endpoint for tokens here too? If not do you have a way to handle non-public files?
---
My guess would be that whenever this service is used for real we actually need to deal with all of the details it supposedly abstracted away.
The hard part of blob storage has never been storage, it's all the parts that we imply when we say "blob storage". AuthN, AuthZ, permissions, versioning, backups, querying, partial updates, etc. etc. And for most "simple" use-cases you need one or more of those.
I'm not saying you could have made any of these parts any easier, but I think you pitch them as easier than they could be.
Public API documentation is something many pointed out, so I consider adding some form of it. Appriciated.
Npm package is meant to be portable. It has 0 dependencies, and only utilizing API's that is widely available in any runtime.
And yes. you nailed it. It's backed by R2 and CF workers.
Unless it's S3 compatible it's going to be a gargantuan task to make this successful as _everything_ uses the S3 API.
But best of luck to OP!
wrangler r2 object put test-bucket/file --file=fileaws s3 sync [source] [dest]
This is the user guide: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcom...
ACLs, for example? How do you control who can access a file you uploaded?
Also, it costs about 20 times as much as Backblaze B2.
If you're a busy indie hacker who just want to be up an running with file storage under 5 mins and forget the whole thing even exists, you can use FILE0.
S3 is stuffed with features for a reason. It's the backbone of the internet so it needs to cover 100% of the use cases and more.
Willing to try wasabi next time.
All in all, I had the impression that while a lot of things are WIP, Wayland had taken a more holistic approach because it was able to learn from X11's mistakes. Has Wayland really that unexpected complexity that it's now as much a part of the problem than it is a part of the solution?
With async programming there is little load in forwarding it on, and that’ll be fine at low scale most of us use.
Probably a great option for profile pics to avoid upgrading your postgres instance to a costly one.
Package has no README https://www.npmjs.com/package/file0 nor links to a repo
Searching Github doesn't give anything (nor any public projects using it).
Installing the package from NPM just gives you a completely minified JS that's pretty difficult to understand https://www.npmjs.com/package/file0?activeTab=code. No need to publish minified JS to NPM imho. Actually reading/stepping through dependencies is always exceptionally valuable.
I should be able to store my data where I want to. If you wanted to wrap a service around S3, Backblaze, or whatever, maybe sell the convenience instead of selling storage?
So it's not a 1-1 comparison.
If you don't see the value in this extra convinience you should stay with your current service! The goal of FILE0 is not to replace these big players, but to provide an option to folks who just want to get things up and running quickly and don't want to think about infrastructure.
Lets say you become a huge hit and megaupload or vimeo or someone else with massive traffic (but not on the "we built our own datacenters and ISP CDN" level) start using your service. What then? Cloudflare will call you and demand you step up to an enterprise plan or pay for premium traffic.
Your pricing model assumes the pricing model of another pricing model that assumes pricing negotiation when usage increases. You will be the middle party in a discussion where you really don't control either party.
This is why I actually like AWS:es nickle-and-dime for every compute second and megabyte.
- Can I trust you?
- Where are your company credentials?
- What's the business continuity plan?
- What's your support? Sending an email to hi@ may not work, when my application is dead because of some bug at your end.
Perhaps a 'mirror to S3-compatible store' feature would address these concerns though. Sure it is extra cost, but it would be a nice de-risking option for early adopters.
Yeah in one way people were much more casual and curious. But keep in mind everything back then were monoliths. You’d just upload files to your server. Some of the complications of trust comes from the fact that you need microservices with custom API credentials and usage quotas in order to run leftpad at web scale.
If you zoom out and think where we are today it’s absolutely insane that uploading a file 20 years later is a closed source subscription service. Nothing against OPs project, but goddamn look at us.
Nevertheless, its nice seeing crappy aws products getting fixed.
Obviously it's a brand new product, with 0 reputation. As with every new product at the begining I expect bugs and disruptions, but IMO the product is simple enough to get it right quickly. FILE0 a simple layer on top of R2, so for reliability and HA is majorly dependant of their system.
You can trust me as much as any random guy on the internet.
For a big cloud provider you will be just a number, they don't particularly care if you're happy or not, but what I can tell you that for me you're my #1 priority. If you're happy, FILE0 survives. If not all the effort building it goes to the bin.
What's missing from the product is the ability to share apps (aka buckets) with your co-workers. If thats a feature you can live without momentarily, businesses can ask for a custom offer if 100GB is not enough. But they won't be billed after seats or any of that crap. Just for usage.
That all being said, I would love to setup a call and have a personal connection, and I'm here to help with anything you might need. I'll enrich the page with more obvious contact details, because it's a legit concern I'd also have.
Saying “you can’t trust me” isn’t a good look for a SaaS.
It worked out for Facebook.
I’m kinda happy about that since we can now use it as an example.
But more importantly, it’s realistic. You should absolutely expect someone to walk away with all your data and zero recourse.
You're mixing up server side and client side.
2. Feature extensibility. This way FILE0 is not restricted by S3 API limitations. One example being: the AWS SDK doesn't support advanced filters for files (like ends with .png). This is a feature FILE0 has, but not supported by the s3 api.
3. If the goal is to provide a smooth DX, you cannot start with "Go to your aws account and create a bucket, and add this configuration to your bucket".
A package like this would be interesting but it's not what FILE0 aims to achieve.
We built simple blob storage in Val Town, which has the added benefit of not needing an external API key if you're using it from within Val Town: https://docs.val.town/std/blob/
The API is similar: `blob.setJSON(key, value)` and `blob.getJSON`
You will pretty quickly get nefarious and copyrighted material hosted on your servers, you will be asked by law enforcement for access and you will get DCMA take down requests, and scary sounding emails from lawyers.
Just engage a good lawyer, and check for any requirements local laws require you to fulfill pre-emptively.
This looks like an interesting product, but it's missing some key technical details to woo engineers -- both how it's done and how reliable the CDN+service is.
So is `new S3Client({})`. It's unfair to S3 to pass redundant credentials in the sample code.
You do need to consider backups, you don't get high availability or anything like that and you're limited to the amount of storage you can attach to the VM. So it's not the same as S3 at all, but depending on your needs it is an entirely valid solution.
Serving files is a large space and people do very different things there. No single solution will fit all use cases and still remain simple.
That said my honest thoughts are that for my company's simple use case, I set it up once in 15 mins and never really thought about it again, but I can see how someone who hasn't done it before would love this kind of DX.
Couldn't presigned URLs work like tokens? And can't you do filtering with s3 select?
It's been a while since I did anything complicated, or much at all with s3 so I am not claiming to be any authority. Just curious!
Something to simplify a cached piece of data into 1 or 2 lines of code would be nice.
Yes. By default all your files are private and only you can access them. To make a public url you can use f0.publish('filename'); This returns a static url that you can share with anyone, and they can download the file.
This url will stay valid until the file is deleted or unpublished via f0.unpublish('filename');
If you call publish again, it will generate a new public URL.
workers+r2 is the ideal stack for 80% of problems now.
to everyone complaining, just use workers+r2 directly. cheaper and more flexible.
this is excellent ux around the much smaller burden of learning workers+r2. bravo.
btw darkreader breaks your css.
Is file0 S3 compatible?
No! I believe if you look for S3 compatible storages you find much better options out there then FILE0. R2, Backbaze and others.
FILE0's mission is to keep things simple for the small guys.
If I’m really sure I want objects to be public, make the bucket (or Azure blob store, etc) public and reduce the lines of code without adding another dependency.
It is great to have the File0 code next to the over verbose S3 code.
The S3 code contains a few security operations that are missing in the F0 code.
What is the story on that?
Do you mean "can't be arsed" here?
That kind of pricing seems distinctly unfair.
The reasons for using R2 instead of S3: - Pricing: S3 charges for everything you do. R2 is only for requests and storage. This enables FILE0 to only charge for storage-tiers which is much more understandable pricing model.
- Workers: Good fit for large-scale file-streaming. And the two works great together.
This looks sick! Thanks for building this!
If I were you, I'd dedicate all your efforts on getting into the Vercel integrations page.
That's where I would re-find it, and probably more likely your user base. (scrappy js apps that don't care more than just getting something working and released)
I feel like f0 is a more minimalistic take on file management. More comparable to Vercel blobs than ut. I don't feel these two are comparable though. ut is a nice product if you are looking for those functionalities though.
Sorry, but your code examples are a flat out lie.
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
Really? C'mon. It's YOUR examples on YOUR site.Regardless, that's not even what I'm worried about.
The part where you need a key for this service and don't need any of that for S3.
Sorry, but now that means I have to handle credentials with this service, which is something I like to avoid as much as possible.
Let's put it this way.
I can spin up an AWS instance right now and do
await s3lib.set('image.png');
and it works.But if I do:
await f0.set('image.png');
I also need to find a way to properly handle this extra token.And my issue with that is this:
const s3 = new S3Client({
region: region,
credentials: {
accessKeyId: accessKeyId,
secretAccessKey: secretAccessKey
}
});
You don't need that.But here is the fun part. Here is the AWS code you need to upload a file and have it available.
await s3.send(new PutObjectCommand({Bucket: 'my-bucket',Key: 'image.png',Body: myFile}));
And that's it. Full stop. No key to muck about with.versus
await f0.set('image.png', myFile);
But..., you also need to set the key, so.... SOME_ENV_VAR_FOR_YOUR_SERVICE=howeverlongthisisyouneedtomakesureitgoessomewhere node app.js
And, honestly, we know that's not even the same thing as what I have up there for AWS.Or...
const f0 = new File0({secretKey="howeverlongthisisyouneedtomakesureitgoessomewhere"})
await f0.set('image.png', myFile);
Or... const config = {
secretKey: "howeverlongthisisyouneedtomakesureitgoessomewhere"
};
const f0 = new File0();
await f0.set('image.png', myFile);
await f0.publish('image.png');
Listen, whatever. I'm tired of people being deceptive to try and sell products to programmers. I get it. You are after the lowest common denominator. Someone who actually doesn't know AWS, S3, or anything about Roles. Someone who could set that up in minutes. Fine, I get it. Convenience is real. Just own up to the fact that your examples are purposely bloated on one end and shamelessly thinned on the other end.You literally add extra stuff on the AWS side and remove needed stuff on your side. To me, that's a lie.
About the rest: For s3 you need to instantiate the client, and the only part you're correct about is that the client credentials can be also auto detected from env vars.
Let's boil the rest down: - For S3 you will need to add 2 env vars: AWS_ACCESS_KEY_ID, AWS_ACCESS_KEY_SECRET. In case you don't have other AWS keys in your env for other services this will be auto detected by the client. (Still needs to be instantiated), so you can cross off the 2 lines for creadentials.
- For FILE0 you need to add 1 env var: F0_SECRET_KEY. Then import the client which is autodetecting your env. ``` import {f0} from 'file0';
f0.set('myshit.png', myFile); ```
I'll let everyone be the judge of which one is simpler for them. And you should also use whichever you like more. I will sleep good at night either way and keep using file0.
I do think the storage "space" could use some disruption. I use s3 every day at work, and my general feeling is that the entire permission system is more complicated than 99% of apps actually need, to the point of being dangerous; I think the complexity has absolutely contributed to so many people getting it wrong and leaking data.
An alternative where the only options were public and private, exclusively set at the bucket level would be good enough for 95% of users and simple enough to rarely get wrong.
AWS went from "just push a couple of buttons and you are running in the cloud without a dedicated sys admin" to "you are going to have to hire a team of cloud admins because no one understands all the options and costs anymore".
Compare AWS to DigitalOcean, for example - the difference in simplicity is mind-blowing.
The latter will really get your customers torqued, the former is going to make some customers not get exactly what they want.
Maybe in the early days this would have made sense, but the cloud vendor lock in is so strong that I seriously doubt most customers even have the leverage. "Add this feature or we are walking to Linode"?
I don't think so. It just seems like a broken product design culture, where the managers need to add features non-stop, so they can make some slides and get their raises. Big company problems, to be sure, but still disfunctional.
Does it cost anything to not use features you do not need?
Last time I checked, all the so-called niche features were stashed in hierarchical option lists, outside of the happy path. You need to purposely want to dig into, say, file access, bucket access metrics, storage strategies, object versioning, etc to actually get to them.
It costs time whenever some coworker asks for something impossible, and says something like "Have you really checked all the options?"
Why do you have team members toggling settings at random at will?
If that's a real problem with your organization, you have far more pressing problems than the number of options offered by a service you consume.
I guess you'd be the type with unrealistic demands.
So, to be fair, yes it costs to not use features you do not need in this case.
You literally pay by the hour to use S3, and you also pay the hours you spend trying to understand the permissions and modify settings, so it literally costs to not use the features you do not need in this case. I'm trying to make an argument, I'm not saying you pay much, but I answered your question, and you do pay, either by time, or money.
Not really. In order to set bucket request metrics not only you need to explicitly set your bucket to be open to the world but you also need to dig down the options to explicitly enable them along with a metrics filter of what objects you cover.
Object versioning is disabled by default and you need to go way out of your way to enable them at a specific level.
You also need to go way out of your way to set another object storage class.
None of these features are enabled by default. You need to turn them on and configure to start using them.
Of course you still have complexity as you have the bucket owner, object owner and requester which might be 3 different entities, but still mentally easier to grasp than policies with dozens of options you need to read documentation for to understand what they are for and what the consequences of using them are.
And then policies are an attempt (I think) at simplifying permissions to a bucket with one declaration, but in practice what I see are people copy-pasting decades-old (really!) policies from other projects because most of us just need to get a private file store up and running.
Platform exhaustion - I have to know so much about AWS to not make a huge mistake that it takes away from my actual product development work that my company expects out of me.
Quick alternative:
#!/bin/bash
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <file-to-upload> <s3-bucket-name>"
exit 1
fi
FILE_TO_UPLOAD=$1
S3_BUCKET=$2
S3_OBJECT_NAME=$(basename "$FILE_TO_UPLOAD")
aws s3 cp "$FILE_TO_UPLOAD" "s3://$S3_BUCKET/$S3_OBJECT_NAME" --acl public-read
if [ $? -eq 0 ]; then
echo "File uploaded successfully."
PUBLIC_URL="https://$S3_BUCKET.s3.amazonaws.com/$S3_OBJECT_NAME"
echo "The file is publicly accessible at: $PUBLIC_URL"
else
echo "Failed to upload file."
exit 1
fi