Webdis: HTTP + JSON API for Redis | Hacker News Reader

Webdis: HTTP + JSON API for Redis

thechangelog.com

55 pointsadamstac29 comments

Memcache, for example, is explicitly not meant to be run with "public access", as it is not secured against attacks. I am pretty sure the same applies to redis.

If my assumption is correct, i wonder why one should use a (potentially slower) http client or protocol in favor of the "native" protocol.

Hello, good point, but I think the idea of the author (accordingly to what I read in the Redis maling list) is to provide some kind of access control list.

I think the most interesting practical application of a Redis HTTP interface is accessing your Redis database directly from Javascript.

One of such simple ACL is to deny all the commands but the few you use, and use unguessable key names. This is good for a low level of security. There are of course much better ways... but I'm curious about how this could evolve.

(I'm the author of Webdis)

As antirez pointed out, there are indeed ACLs in Webdis. You can enable or disable commands using a CIDR match and/or HTTP Basic Auth. (ex: disable all write commands for everyone, but enable SET for authenticated clients on the local network).

Another use is for obtaining an async redis connection using async http frameworks that may not include a redis client, e.g. tornado. I'm doing something like this at the moment using tornado's async http client talking to a nodejs http server that is basically proxy'ing some simple redis commands.

What is the primary practical benefit of this?

It seems to simplify doing some really cool things, but I'm not sure what those things are...

To me the main benefit would be that it allows you to plugin other technologies: caching via varnish, proxying va HAProxy, authentication for APIs, etc. because HTTP is well supported.

Also, you can access redis via javascript---client or serverside.

One benefit is that Javascript can interface with Redis directly through AJAX requests.

Redis doesn't have a native http interface. Having this kind of interface open all kinds of possibilities for working with Redis. For example, I've been doing some work with Node.js and Redis and one side project I worked on for a while was an http interface. One of the awesome things about CouchDB is its http interface.

One would be that you can access Redis servers with apps that don't have good / mature Redis clients - like Adobe Coldfusion. Even though you still have to cope with the buggy JSON support in Coldfusion then. But better than no Redis at all i guess.

Good question :)

I started Webdis because I thought it would be a cool project, after seeing suggestions about such an interface on IRC #redis.

Other that the cool factor, the main benefits are IMO:

* Javascript clients can access your Redis base. This is already done by CouchDB, on a much larger scale. Webdis is nowhere near such capabilities.

* A “Comet” interface to Redis pub/sub commands to stream data using either HTTP chunks (or WebSockets, not implemented yet).

* A way to query your Redis server without having to write code. Point your browser to Webdis and see for yourself if the 12th element in that list is what it should be. Having a public interface doesn't mean you have to expose it to your clients, it can be for internal use.

* A remote access for heavy clients such as Flash or Air. Webdis already serves a crossdomain.xml file for Flash clients; they can query Redis themselves instead of going though a proxy that you'd have to write by hand.

There could be other ways to use it; I'm glad to see the project gain some traction, it will help me figure out what's really important.

Very good points.

Actually that's something i started doing too with Node.Js, an HTTP interface to Redis for my personal use.

At first i did go with a RESTful approach, but you quickly hit the limit of commands available versus the numbers of commands in Redis.

The choice i did was the following:

Read command are accessed through GET this way:

GET http://localhost:8124/get/mykey

200 {"mykey" : "mydata"}

GET http://localhost:8124/hgetall/mykey

200 {"hashKey": {"field1": "value1", "field2": "value2"}}

Write commands are run through PUT or POST :

POST http://localhost:8124/set/key/value

201 {"response": 1}

PUT http://localhost:8124/lpush/myList/1

201 {"response": 1}

Delete commands through DELETE.

DELETE http://localhost:8124/del/mykey 200 {"response": 1}

DELETE http://localhost:8124/hdel/myHash/field

I'm pretty happy with the result, there are some commands i'm not sure where to put though(lpop, rpop, multi, ...)

Why the wacky SET/GET? Why not being restful and using HTTP verbs?

Probably since in Redis there are a multitude of write and read operations, so anyway a good HTTP protocol "map" is not possible.

I agree, why on earth would somebody set something with a GET request?

If you separate the syntax from the semantics, you'll see that the GET request (syntax) is merely the mechanism of accessing the appropriate function (semantics). If you reread the short article after deleting all the junk about REST and CouchDB, and ignore the presumption that this is a RESTful API, then the article is clearer: Webdis is an HTTP interface to Redis.

Hmmm, you're right. The bit about REST and CouchDB is a bit misleading.

Because there are a LOT more commands[1] than "set", "get", and "del".

http://redis.io/commands

So? REST does not limit the amount of operations you can do on resources.

You must've misread somewhere. The OP said "Why not being restful and using HTTP verbs?" I said "because there are far more commands than HTTP verbs".

In other words, this has nothing to do with "the amount of operations you can do with rest", this has to do with not being able to use the limited number of HTTP verbs to correspond to the multitude of Redis commands.

(I'm the author of Webdis)

I agree with you, and to be honest this is what bothers me the most in the current state of the project. The top point in the list of TODO/IDEAS is currently “Add better support for PUT, DELETE, HEAD, OPTIONS? How? For which commands?”.

I'd be very glad to hear about a better way to implement this. As antirez (author of Redis) and kbd pointed out, Redis supports a lot more than GET/SET. How do I make LPUSH/RPUSH/SADD... RESTful?

why SET, maybe POST?

I think any command that modifies and writes to the DB should be POST or PUT (for editing existing entries), what reads should be GET and anything that delete should be DELETE?

Please please please do not make the API dependent on HTTP method verbs. This completely breaks JavaScript apps[1] and it's an arbitrary constraint that does not benefit anything except REST zealotry.

[1] For example, cross-domain JavaScript requests can only read GET responses.

I think there should be a way to disable GET for some commands, this seems to be a serious concern. That said, you'll always be able to access Webdis using GET; you present a common case that justifies the existence of such a feature.

I have started to implement Cross-Origin Resource Sharing, which should enable cross-domain JS requests using POST (If I understood correctly). An OPTIONS call is made by the client before the POST.

Hi thamer, your question made me curious so I decided to see what I could come up with. I've never used redis so this might be all whack, but here's what I did with the list type after looking at the redis docs:

Type operations:

POST /list?key=<key>&value=<value> -> LPUSH

DELETE /list?timeout=<timeout>&key[0]=<key>&key[1]=<key> -> BLPOP

List operations:

POST /list/<key>?value=<value> -> LPUSHX

DELETE /list/<key> -> LPOP

Search based operations:

POST /list/<key>?find=<value>&value=<value> -> LINSERT BEFORE

DELETE /list/<key>?find=<value>&count=<count> -> LREM

Index based operations:

GET /list/<key>/<index> -> LINDEX

GET /list/<key>/<index>?count=<count> -> LRANGE

PUT /list/<key>/<index>?value=<value> -> LSET

Transactional operations:

POST /list-tx/rpop/push?source=<key>&target=<key> -> RPOPLPUSH

POST /list-tx/rpop/push?timeout=<timeout>&source=<key>&target=<key> -> BRPOPLPUSH

Length

GET /list-length/<key> -> LLEN

Reverse operations:

POST /rlist?key=<key>&value=<value> -> RPUSH

POST /rlist/<key>?value=<value> -> RPUSHX

DELETE /rlist/<key> -> RPOP

POST /rlist/<key>?find=<value>&value=<value> -> LINSERT AFTER

DELETE /rlist?timeout=<timeout>&key[0]=<key>&key[1]=<key> -> BRPOP

Hi David,

Interesting take on it, your HTTP verbs seem to be pretty well used. I'm not sure that expressing commands with a query string would be very easy to remember though. The advantage of /COMMAND/ARG0/ARG1.../ARGN is that you can pretty much type a redis command like you do in telnet, replace spaces with slashes, and have an answer.

I also maintain the phpredis extension for PHP, and having method names that differed from Redis command names has annoyed people a lot (e.g. lGetRange instead of LRANGE). phpredis now has method aliases with the same name as the underlying Redis command so that users don't have to go back to the documentation all the time.

Some people mentioned that GET should always be usable for Javascript apps. It could be interesting to create a list of commands with their respective verb recommendations, and send that recommendation with the response. A strict mode could also for the right verb and disable write commands with GET requests.

Thanks for those suggestions!

Only LINSERT, LREM, LRANGE, and the blocking commands are expressed using the the query, but I agree that none of them should be. The others are just arguments which have to be arguments in order for the resources to work. Which leads to...

Keeping the Redis names means REST isn't even an option. There's no way around that. For instance LINDEX and LSET operate on the same resource, i.e. the same URL. Giving them different URLs means abandoning any and all RESTfulness.

This exercise was probably only useful in showing that REST is not what you want, but it was interesting to think about. Thanks for the question!

There's also Bone which implements SET, GET, and KEYS over HTTP(S). It also uses something similar to the Amazon S3 request signatures for authentication.

https://github.com/solutious/bone

This is also called "CouchDB."

Reply on news.ycombinator.com