Jsonstore.io – Store your data for free without signing up | Hacker News Reader

Jsonstore.io – Store your data for free without signing up

jsonstore.io

136 pointsbluzi53 comments

So this is the same what Pastebins are for unstructured texts and image hoster for pictures. At one point, at least everybody realizes that JSON can host arbitrary data -- such as unstructured text -- and pictures as containers, can do as well.

I wonder how the operators of these free services act in terms of durability. https://jsonbin.io/ proudly reports: "21838 bins created". That probably does not need much storage. But what if I use it regularly from my systems? I'm not so much convinced of just another service which allows storing data for some time, probably with a nice UI.

(Actually I noticed this with http://sprunge.us/ -- I included uploading a build log into my build systems which my team is using now. At one point, sprunge.us started to give back 500's for us, basically stopping working. Sure, switching the service takes a blink of an eye.)

JSONbin.io developer here. Tracked the thread using GA after unusual spike in the traffic.

Good point. We cannot trust any random service. It takes some amount of investment, dev time & server costs to keep the service up and running. The reason I developed this service is to ease the storage process for the developers developing mobile or small scale apps, so that they can focus on the app rather than spending time setting up the database. Just to keep the website up and running, I'm going to introduce nominal amount plans which developers can afford and it pays my server bills. I'll ensure that I've SLA in place when am introducing pricing. The only reason I've not introduced pricing on the website is I don't want to charge for something basic what am providing for now. Will add it when I provide more powerful features which are already under development.

What's your take on developers not putting the effort forth to use their own object store (ie S3) for this?

I'm not against developer velocity (disclaimer: not a dev), but this seems to be a cycle where hosted tool comes out, people rely on it, tool becomes too expensive to run, neglected, etc, and then tool goes dark one day. Would time be better spent on client libraries that ease the difficulties of using existing durable storage systems? Or open source javascript frontends you can drop into your S3 bucket to provide a GUI for manipulating JSON in the same object store?

I am seeking enlightenment, not an argument. In tech for almost 20 years now, and have seen my fair share of the cycle.

It's not just about efforts but it's more like the developer needs to maintain another piece of a service. I agree that uploading a simple Json file doesn't sound too complez to maintain but at a cost of you paying for S3 service with limited amount of features.

> What's your take on developers not putting the effort forth to use their own object store (ie S3) for this?

Services like these are really geared towards the hacker mentality of getting prototype built as quick and dirty as possible. As the parent wrote--to focus on the app.

Using a durable source like S3 sounds good, but now you have the additional requirement of having an AWS Account. Granted the barrier to getting an AWS account is minor, but it is still is a barrier.

I believe your question is equivalent to the question of solving the tragedy of the commons, which suggests it's indeed a tricky problem.

As a strawman proposal, we could take some of Elinor Ostrom's suggestions for a non-governmental solution (https://en.wikipedia.org/wiki/Tragedy_of_the_commons#Non-gov...) and apply them:

1. Introduce the concept of an API key to allow users to voluntarily segregate the traffic into accounts. API keys are only voluntary because it's easy to extract someone else's key from a client binary and masquerade as that person.

2. Issue API keys only to users who agree to a set of community standards.

3. Nontrivial usage that still complies with the community standards requires payment. The payment should be structured so that it's reasonable for medium-traffic users to stay, but more economical for heavy-traffic users to leave the community and start up their own clone of the service. For style points, the community standards should have specified that anyone who starts with the free community service and graduates to hosting a clone must also host low-traffic free users, basically causing the service to become federated. Maybe the original service replies with 301 redirects to the new service so that old clients keep working.

I think all of this can be fully automated, and if you're cool accepting Bitcoin for usage above the free tier, then you don't even have to worry about how to handle payments. There are plenty of VPS providers that accept Bitcoin these days, so the system is even closed-loop from the perspective of the service operator.

I built essentially what you describe a while back (jsondata.io if you're interested, though the certificate has expired and I've turned off the actual service for now). The idea was to make an infinitely scalable (within reason) solution that works well as a pastebin-style, no-setup-required data store for hacking on projects, but also as something that can scale with the needs of a business.

The architecture is built on AWS Lambda and DynamoDB, along with a small Ruby Sinatra service to handle payments and access control.

After running it for a while and getting some limited feedback from potential users/customers, I ultimately abandoned the project due to an apparent lack of interest. For hobby projects, it's fairly easy to spin up a self-hosted data store for low volume needs (or use one of the other free services). For anything more than that, a business is probably going to spend the necessary time/money to maintain a full DBMS or use something like Firebase.

I might be missing an opportunity (I've had thoughts about resurrecting the project), but for the time being, I'm just not convinced there is enough demand on the paying side.

Part of the fun of prototyping is setting up your own service; at least, I always find that part fun. That likely eliminates some of the target market.

If the service API were fully standardized, and users paid for their own database, and signing up for a new website meant that the user granted controlled DB access to the site, then that might be enough impetus for developers to use a provider of the service. Maybe it could follow a Dropbox model where the first N bytes/month are free per user, and service providers pay affiliate fees to sites that recruit lots of paying users.

So, my first thought is that I'd set up a service like yours primarily because I'd be fascinated by the data provided. While that might not result in malicious use of said data (it'd not be my intention), it's quite possible the resulting data might provide interesting opportunities.

Now, I'd consider it unlikely that I'd setup such a service without being a peeping Tom, so if something interesting were to come my way, I'd probably notice it.

How do you deal with this? A firm resolve not to, at the very least, check out shady avenues of information? Giving in to that urge but not using it for profit?

The reason I'm wondering about this is that I'm part of multiple IRC channels where with some regularity people will link to a service such as yours with code that exposes private keys, and while usually the people helping out in those channels will be quick to point out the privacy leakage, it struck me that all of it is still available to the hoster. And in fact, switft removal of said code my ping the hoster that this might be interesting data.

To make it clear:

- jsonstore.io is intended for small projects, mostly for the open-source community. It's not a datastore you can use to store all the logs of your company, and we do have quota limits.

- We do nothing with your data, it's secured and if you want to store sensitive data you may want to encrypt it.

- The idea of this project is to provide the open source community a easy solution for storing casual usage data/settings for their projects.

Just want to drop a big thumbs up for making the project open source unlike mine. It will really help developers who are looking for an API and want to host it themselves. I'll try to contribute to your project in the best possible way I can. Cheers.

Thanks, appreciate it. :)

> - We do nothing with your data, it's secured and if you want to store sensitive data you may want to encrypt it.

You mean "it's NOT secured" right?

Hashing a value from Math.random() with a cryptographic hash (i.e. SHA256) doesn't make it cryptographically random[1].

If you want a random string get one directly via crypto.randomBytes(...)[2]:

    const id = crypto.randomBytes(32).toString('hex');

[1]: https://github.com/bluzi/jsonstore/blob/87af0d3ef6bf11222b98...

[2]: https://nodejs.org/api/crypto.html#crypto_crypto_randombytes...

Thanks, feel free to create a pull request.

Why not use uuids?

These need not be cryptographically random either.

UUIDs are fine too. What matters is how they're generated.

If you're generating v4 UUIDs server side using the "uuid" NPM module then you're fine as internally it's using crypto.randomBytes(...)[1] with an almost 16-byte random string (UUIDs are 16-bytes but a proper v4 UUID has to override some of the bits to conform to the spec[2]).

If you're rolling your own UUID function or generating them client side then they may not be as random as you think. For example the same uuid NPM module silently uses Math.random()[3] on the client side if it can't find a better alternative. It's fine for something purely local to the one browser but I wouldn't rely on it being unique globally.

[1]: https://github.com/kelektiv/node-uuid/blob/17d443f7d8cfb65a7...

[2]: https://github.com/kelektiv/node-uuid/blob/17d443f7d8cfb65a7...

[3]: https://github.com/kelektiv/node-uuid/blob/17d443f7d8cfb65a7...

Sounds like a vaguely similar project to RequestBin[0].

Just a word of warning: "We have discontinued the publicly hosted version of RequestBin due to ongoing abuse that made it very difficult to keep the site up reliably."

[0]: https://github.com/Runscope/requestbin

So there's no actual security on this? That is, given the knowledge of an endpoint, anyone can issue requests against it?

How do you plan on responding to requests regarding GDPR requirements from EU residents? What happens when someone starts uploading content that could be covered under the DMCA? Following the links, should we assume that https://www.forter.com is the responsible company?

Yet Wikipedia exists.

Yes, and they have to comply with the DMCA and GDPR as well.

Not at all, it has nothing to do with Forter. It's my own personal project.

Ok, that's fine, but it doesn't explain how you'll ensure that GDPR requirements will be handled.

Based on this I think there’s no obligation: “...a non-EU company with a website accessible to people in the EU is not, on its own, sufficient to require the company to oblige with the GDPR.” From: http://emerging-europe.com/voices/preparing-for-gdpr/

The number of people invoking GDPR on HN indicates that a lot of people are going to be let down very soon.

I hope you've learned a lesson about creating things that may help people on the modern, legislated internet. Next time, be prepared to handle all complaints first or stop providing services to people and let the bigger companies do it that can absorb the administrative costs that come with worldwide legal conformance.

It's quite sad that this is the current state of the internet.

Looking at your source code, it appears you're wrapping firebase. I'm very curious to hear how much this will cost you after the HN traffic hit.

Be sure to check out webdis ( https://github.com/nicolasff/webdis ) for a similar service you can run on your servers. It's all in C so it's superlighweight.

Thanks for sharing! fyi: You can also run jsonstore on your own servers.

Just enter the website, copy your link, and start sending HTTP requests to store data.

GitHub: https://github.com/bluzi/jsonstore

The curl DELETE example is wrong (it's identical to the GET example).

JavaScript developers interested in this should also take a look at https://encapsule.io/docs/ARCcore/filter that makes it simple to implement bulletproof JSON validation and normalization. Advanced application of this library additionally allows for runtime type-based metaprogramming in JavaScript (i.e. build generic algorithms that are specialized to handle specific app-defined data types at runtime).

It even has an automated assistant...who looks like he's about to beat you up and take your lunch money.

How do you deal with abuse? E.g. people posting (links to) copyrighted stuff, or large amounts of data?

Since we are talking about the value of an online JSON storage system, would any of you be interested in a collaborative system for common JSON data sources? Something like Wikipedia but for structured data that currently lives as files or github repos?

"Create a microservice that stores JSONs." Says the senior SE to the intern.

Another similar tool (FOSS, source is on GitHub somewhere): https://jsonblob.com/

It also has the added advantage of a JSON editor with tree view.

"This site is blocked due to a security threat."

First time I'm seeing this message so I doubt it's a case of over-sensitive Cisco firewall. Chrome also blocked it as a threat.

Cisco firewalls block GitHub occasionally...

If i were to host this locally, what are jsonstore benefits compared to let's say couchdb (opensource, stores and serves json documents over http)?

Basically a https://jsonbin.io clone?

Sounds like it, though as the author of jsonbin is in this thread points out, this one is open source: https://news.ycombinator.com/item?id=16765885

gotcha

How does this differ from Firebase?

It is essentially a think wrapper on Firebase:

https://github.com/bluzi/jsonstore/blob/master/api/database....

Cool idea! I'm kinda jealous I didn't think of it myself. Very simply but useful.

Nice work!

what is the backend / infrastructure like ? is it just Mongodb?

Feel free to look at the sources: https://github.com/bluzi/jsonstore

This looks handy! Thanks!

this just opens up a whole new world to static websites!

Love the implementation! Would be great to eventually see pagination and parameter/ID filtering (/users?age=32, etc)

Reply on news.ycombinator.com