Volatile – A key-value pair API | Hacker News Reader

Volatile – A key-value pair API

volatile.wtf

64 pointsnewbold73 comments

> Create a key-value pair

> GET /?key=key&val=value

GET is supposed to have no side effects, according to RFC2616 9.1.1.

I'll allow it.

My initial response to your comment was "but then the page requires JS", which of course is not true, since a simple inline form could POST and then a redirect could get you back.

I'm no webdev, I think JS has been hyped so much around me I forgot plain html can actually do things...

The correct response is "but then I need a webpage/separate tool".

When all API are GETs, the browser, which is likely to be open when you read this, is all you need.

> When all API are GETs, the browser, which is likely to be open when you read this, is all you need.

And your service is also incorrect due to the issues with side-effects mentioned elsewhere in the thread.

It depends on how you interpret correctness. RFC 2616 9.1.1 also used "SHOULD NOT" and not "MUST NOT", indicating that there are valid use cases. The issue is about whether your users are aware of (and are responsible for) the side-effect or not. In this case they are.

Except when GET has side effects, one can embed the URL in an img tag and visitors would be triggering that side effect unknowingly. One can do this on most public forums. So no, users may not be aware.

What you just described is actually a web beacon [1], in which case the one that embed the URL is the user (and not the one who load the <img>). Web beacon can be used to implement useful thing like visitor counter service [2].

[1]: https://en.wikipedia.org/wiki/Web_beacon

[2]: https://www.hitwebcounter.com/

It's also "useful" for cross-site request forgery.

Cross-site request forgery is only "useful" in auth context. There is no auth in Volatile, nor in the hit counter.

Contrary to popular belief, the browser can perform POSTs as well as GETs.

An inline form on the webpage that POSTs the result is not a "webpage/separate tool". Option 2 would be to use the browser's dev console.

You need a webpage to have an inline form, which you need to create somehow. Compare that to just opening a new tab and type in an URL.

The dev console is a separate tool. And I'm still sure it is much quicker and easier to open a new tab and type in an URL, than to do a POST with dev console.

Unfortunately Volatile doesn't seem to support redirects.

Imagine doing a code review where someone wrote a function “getKeyValuePair” and it’s actually a method to post it. No :angry-no-face:

For a real world example of why idempotency is important and not just some random suggestion: https://twitter.com/rombulow/status/990684453734203392?s=21

Judging from the twitter thread, idempotency vs. safety is not widely understood.

The "create a key-value pair" operation described here actually is idempotent.

The requirement on GET is that it must be safe, meaning no side-effects, for a variety of reasons, not just browser behaviour.

The difference is described in detail in RFC 7231 section 4.2, or the older RFC 2616 section 9.1.

Could you expand on this? I've read your comment a couple of times and still don't quite understand. How is it idempotent?

Idempotent means that a given request always has the same result. This GET request fits that definition, since making the same request several times will always return the same thing. The fact that it modifies data means that it’s not safe, but that is a different concern. Remember that PUT and DELETE are also idempotent

Of course it's not just that it returns the same result, but that the request has the same effect. This is not entirely true since the modification date will change, but other than that it's true that setting a key to the same value multiple times is idempotent.

If the modification date is changed, then its idempotency is open to application-specific definition.[1]

One way you might call modification time changes idempotent is if you don't care about modification times.

Another way is much subtler: Although an updated modification time represents a change, the point of HTTP idempotent requests is that repeating them automatically is harmless. This matters because some requests may be repeated automatically by some clients under some network conditions.[2] Updating the modification time twice to a near-current timestamp might satisfy the idempotency condition for the purposes of the application.

[1] This isn't as handwavy as it sounds. It means that because it's the application that determines whether the effects of a request count as "semantically" idempotent (or semantically safe), the application should choose the appropriate HTTP method (or "verb": GET, PUT, POST etc) for the application's needs. The application's choice of HTTP method matters because client libraries, proxies and servers perform different methods in different ways in the protocol.

[2] For example, if sent as the second or later request on a persistent HTTP/1.1 connection, just as the server closes the connection, then the client might retry any requests that are for an idempotent (or safe) method. So it may automatically retry GET, PUT and DELETE but not POST.

> One way you might call modification time changes idempotent is if you don't care about modification times.

My point exactly :)

Idempotent means that doing it once is the same as doing it more than once. This is standard terminology.

The parent comment uses "safe" to mean doing it zero times is the same as doing it once or more. This is not a definition I've heard before, but maybe it's common in this context. A more normal term would be "free of side effects".

"Safe" and "idempotent" have defined meanings in the HTTP standards, that's why I used those terms, and provided the RFC section that explains for anyone who cares to read more.

Right; it's somewhat silly to suggest standard compliance to a service obviously not meant for any sort of serious use, but this irked me too.

> a service obviously not meant for any sort of serious use

Multiple existing and established services were created as proofs-of-concept, silly jokes, experiments, and other "non-serious use" projects. Many of them have survived and we are forced to bear with their sloppy design all the time.

Have you ever used e.g. JavaScript?

I've added a FAQ section to the site and have addressed this concern there.

Just say no to APIs that don't respect RFC2616 9.1.2 It will bite you.

I think a worthwhile addition to this service would be the ability to get keys by their hashes. That way, you could separate read-only and read-write key while still keeping it simple. I can imagine a program offering i.e. update checking that way.

> Wait, didn’t that key belong to someone else? No, because that’s not how things work here.

Given the lack of auth, it should be possible to overwrite other people's keys, or did I miss something?

Yes, that is how I understood this as well. https://volatile.wtf/foo previously read "ok boomer" but now it reads "bar".

What a fascinating artifact for future archaelogists. Or cultural anthropologists?

There needs to be something to dig up for archaeologists to be able to find it. We don't know if this service isn't going to just disappear overnight.

"that's not how things work here" means there is no such thing as a key that belongs to someone else. From the next sentence: "no ownership". All keys belong to you as much as anyone else.

> Given the lack of auth, it should be possible to overwrite other people's keys, or did I miss something?

Hence, "volatile".

Really, really volatile.

I made the bigger brother of this, still simple; but with auth (simplest possible) and JSON: http://root.rupy.se

Oh, it has word indexing (sort), relations (link), tree structured meta-data (meta) and it's distributed!

Also it's completely async. concurrent (joint parallel) so it has zer0 IO-wait! :)

What do you mean by async concurrent and parallel? How is it more asynchronous than regular http?

HTTP is a protocol without specification when it comes to async./sync.

Most HTTP is sync. because non-blocking IO and concurrent memory access are hard to make without side effects that our society does no not like; like dropping slow clients or using locks and chasing threading bugs f.ex.

My web/app-server is one of the few that does it all without bloat: parallel + async. with concurrent data structures and non-blocking IO; it means all cores can cooperate on the same problem at the same time without waiting for anything.

IO-wait is where most current sync. server solutions loose ~20% of the CPU when a system is under high utilization.

Comparable to what you loose on the kernel which is what many people are trying to remove with user-space networking.

It's also because of IO-wait that services break, a completely Joint Parallel system is very hard to break, it gets increased latency instead.

All Joint Parallel systems have a queue built in because there is no point to spawn more threads than there are cores.

I'm trying to find a name for it because things usually needs a name to spread, and Joint Parallel is the best I got so far.

Neat, would come handy for demos and personal learning projects.

What is the tech behind this ?

To a reasonable scale, a redis instance behind a small web server should do this for you.

I built a similar one in the past using dynamodb and aws lambda. I don't have to pay a dime when I am not using it.

https://thesimpleapi.com/keyvalue

This is really cool (and so are your other projects)!

Thanks!

Nice to have more options for minimal key-value APIs, but support for only GET requests and plain text data could be limiting for some apps.

Shameless plug: if anyone's looking for a key-value pair API with a more rich API with support for buckets, access control, and server-side scripts, check out https://kvdb.io (disclaimer: I built it)

Since they naively count unicode characters, https://github.com/qntm/base65536 is the best data encoding mechanism. You can fit 15 bits per character, or a total of 510 bytes per key and value.

This sounds crazy and awesome, I love it.

Not working. Getting 404s upon creating...

    :~$ curl -v https://volatile.wtf/?key=65cd62ba-274c-4844-9ce4-c5d8bedcb959&val=baz
    [1] 6628
    :~$ *   Trying 138.197.77.98...
    * TCP_NODELAY set
    * Connected to volatile.wtf (138.197.77.98) port 443 (#0)
    * ALPN, offering h2
    * ALPN, offering http/1.1
    * successfully set certificate verify locations:
    *   CAfile: /etc/ssl/certs/ca-certificates.crt
      CApath: /etc/ssl/certs
    * TLSv1.3 (OUT), TLS handshake, Client hello (1):
    * TLSv1.3 (IN), TLS handshake, Server hello (2):
    * TLSv1.2 (IN), TLS handshake, Certificate (11):
    * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
    * TLSv1.2 (IN), TLS handshake, Server finished (14):
    * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
    * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
    * TLSv1.2 (OUT), TLS handshake, Finished (20):
    * TLSv1.2 (IN), TLS handshake, Finished (20):
    * SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
    * ALPN, server accepted to use http/1.1
    * Server certificate:
    *  subject: CN=volatile.wtf
    *  start date: Nov 11 19:13:05 2019 GMT
    *  expire date: Feb  9 19:13:05 2020 GMT
    *  subjectAltName: host "volatile.wtf" matched cert's "volatile.wtf"
    *  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
    *  SSL certificate verify ok.
    > GET /?key=65cd62ba-274c-4844-9ce4-c5d8bedcb959 HTTP/1.1
    > Host: volatile.wtf
    > User-Agent: curl/7.58.0
    > Accept: */*
    > 
    < HTTP/1.1 404 Not Found
    < Date: Tue, 12 Nov 2019 07:40:51 GMT
    < Server: Apache
    < Content-Length: 0
    < Content-Type: text/html; charset=UTF-8
    < 
    * Connection #0 to host volatile.wtf left intact

worked for me - you need to quote the url (& made it run in the background)

  [5 bytes data]
  > GET /?key=65cd62ba-274c-4844-9ce4-c5d8bedcb959&val=baz HTTP/1.1
  > Host: volatile.wtf
  > User-Agent: curl/7.49.1
  > Accept: */*
  >
  { [5 bytes data]
  < HTTP/1.1 201 Created
  < Date: Tue, 12 Nov 2019 07:46:14 GMT
  < Server: Apache
  < Content-Length: 0
  < Content-Type: text/html; charset=UTF-8
  <

Is this the hacky & free & no registration version of cloudflare workers KV [1] ?

1: https://www.cloudflare.com/products/workers-kv/

So this is just a funny thing right. It’s not really useful for anything?

I mean, I could see myself using this as a way to store a string somewhere for later.

But someone else can just overwrite it anytime without you realising?

Correct.

use an unguessable string as your key. make it as safe as a password and it might as well be one.

It depends.

Unless the author took preventive measures to prevent this, key lookups tend to be vulnerable to timing attacks, for example.

I use a plain text file for this.

  curl -i "https://volatile.wtf/?key=foo&val=%F0%9F%92%A9"

  HTTP/1.0 500 Internal Server Error

Suggestion: offer a version of this service which forces keys to be sha256 hashes of the values (i.e. content addressing)

Why not do it yourself?

Because I don't have time to run such a free service. The author of Volatile is already doing it, I'm just suggesting a small variation in the theme that would solve the problem if multiple users potential overwriting each other's values (only a server side check can ensure that)

I could be wrong, but I think parent post suggests to compute the hash yourself and use Volatile as is.

ah; because another user could overwrite this value breaking the invariant. If the invariant is maintained server-side, then multiple users can gracefully coexist (without auth)

How is this gonna prevent me from consuming all 1000 requests/hour and prevent everyone else from using it?

I think you are misunderstanding the limit. It looks like standard perip sort of rate limiting rather than per key.

it could be fun to write meta games on top of this.

example: in the `and_then` key, a user (over)writes the next line in a developing narrative. next user repeats. etc. set up a little poller to fetch the key periodically to preserve the history.

I love this idea! I'm logging all changes to data, mostly just for testing/debugging, but I think a "history" endpoint would be handy for this. Not sure what kind of output it should have. Maybe multiple different types, ranging from a plain text dump (one line per change, sorted chronologically) to a JSON object? Not sure.

I'm now getting this site blocked by Cisco security Umbrella :(

Some services block all newly registered domains. I just registered this one yesterday. :(

I hope it starts working for me soon. I just had a very fun idea to make with it.

spent a little time adding this https://volatile.wtf/bee-movie-part-1

Great Idea!

If you wanted to monetize, but still keep it free (of course, you could and should have paid tiers too), you could send back an additional 255 characters of ad-text, with the agreement being, "you use the free service, you are obligated to display the ad-text somewhere, if it's a web app..."

Advertisers pay for the ad-text.

You collect the revenue.

Wishing your service luck!

Caddy doing similar things left bad taste in my mouth, wouldn't recommend this approach to anyone. It's key-value API for gods sake.

While it's just a key-value API it still incurs costs to the developer, there has to be some way for them to recoup that. Whether it's through advertising, or through donations (how likely are people to donate).

I plan to keep it free indefinitely. I'm already using it in two personal projects and I registered the domain for 5 years (a first for me, heh) so it's not going anywhere. If I have time I might eventually roll out some fun features that people might be willing to pay for, but for now I'm just focused on making it easy for people to store, change, and retrieve small pieces of data.

Reply on news.ycombinator.com