Low-level web programming in Racket and a wiki in 500 lines | Hacker News Reader

Low-level web programming in Racket and a wiki in 500 lines

matt.might.net

135 pointsihodes28 comments

I'm glad to see someone talking about using the lower-level web libraries in Racket. The high-level continuation-based ones are ridiculously unsuitable for actual web programming (they store full state on the server at all times, and the URL gives you full access to the session, among other issues), but the lower-level ones actually look pretty clean and modern. And yet all the documentation keeps talking about the continuation stuff, bizarrely.

Indeed. I have fun making little web apps which serve a small single purpose and can be run from a single script[0] - templating using response/xexpr with paredit is.. well.. fun! Speed isn't a problem since I cache the result behind nginx - these aren't mission critical apps.

[0] Tokyo Art Parties parsed out of Tokyo art beat events xml - https://github.com/minikomi/tokyoartparties-rkt/blob/master/...

The continuation-based libraries are the simplest, easiest, quickest way to get a program off the ground. They make interactive Web programming just as easy as writing “scanf” or its equivalent, _and_ are safe in the face of various browser interactions. Therefore, they're a good default for prototyping. However, I agree that they also have issues, and the documentation should not over-emphasize them, but rather indicate what they are suitable for and what not and make clearer to readers a transition path out of there.

I think that depends on where you're coming from. If you're a longtime web developer, stateless request/response are in your blood, and stateful, persistent programming is deeply weird.

The issues you mention are all alleviated by the stateless continuations which do not store state on the server and allow various ways of closing off the session from simply having the URL.

The documentation discusses the features of the Web framework, many of which have to do with its unique aspects. As you can see in this post, all normal Web programming is possible and easy, but there's very little special to say about it, because it's just a normal program.

"It's just normal" is a thing that I think is good to communicate. Being able to use Racket to write web apps in ways that wouldn't be unfamiliar to Node devs makes the Lispy thing seem less weird and intimidating.

This is very useful. I'm in the middle of a Racket web project right now and I got hung up on parsing/processing form data. The Racket docs are very thorough, but they can be confusing if you don't know what you're looking for.

If anyone here has experience with web stuff in Racket, I've set up a small skeleton for Racket web projects: https://github.com/samertm/web-project-bootstrap.rkt

Right now it contains skeletons for talking to sqlite, setting up a router and handlers, and manipulating the web server from the command line. I need to add an ORM (I'm not sure if Racket has any up-to-date ORMs?) and static file handlers. I'd love help or feedback!

From you experience, how is performance? what would you compare it too? The examples ae useful, thank you.

I haven't written any big projects in Racket, so I can't say how it fares on real-world projects with real-world data, but here are the results for a very minimal Racket project using ApacheBench:

~ $ ab -n 100 -c 10 "http://localhost:8888/"

This is ApacheBench, Version 2.3 <$Revision: 1638069 $>

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software: Racket

Server Hostname: localhost

Server Port: 8888

Document Path: /

Document Length: 89 bytes

Concurrency Level: 10

Time taken for tests: 0.084 seconds

Complete requests: 100

Failed requests: 0

Total transferred: 26800 bytes

HTML transferred: 8900 bytes

Requests per second: 1192.32 [#/sec] (mean)

Time per request: 8.387 [ms] (mean)

Time per request: 0.839 [ms] (mean, across all concurrent requests)

Transfer rate: 312.05 [Kbytes/sec] received

Without a baseline comparison against other servers with same functionality on the same computer, these results mean nothing.

For example here are benchmarks from Django's single-threaded debug webserver:

Benchmarking 127.0.0.1 (be patient).....done

Server Software: WSGIServer/0.1

Server Hostname: 127.0.0.1

Server Port: 8000

Document Path: /

Document Length: 2 bytes

Concurrency Level: 10

Time taken for tests: 0.100 seconds

Complete requests: 100

Failed requests: 0

Total transferred: 16400 bytes

HTML transferred: 200 bytes

Requests per second: 998.90 [#/sec] (mean)

Time per request: 10.011 [ms] (mean)

Time per request: 1.001 [ms] (mean, across all concurrent requests)

Transfer rate: 159.98 [Kbytes/sec] received

Or Nginx returning the same thing:

Benchmarking 127.0.0.1 (be patient).....done

Server Software: nginx/1.6.2 Server Hostname: 127.0.0.1 Server Port: 9080

Document Path: /gangam Document Length: 14 bytes

Concurrency Level: 10 Time taken for tests: 0.013 seconds Complete requests: 100 Failed requests: 0 Total transferred: 15600 bytes HTML transferred: 1400 bytes Requests per second: 7478.31 [#/sec] (mean) Time per request: 1.337 [ms] (mean) Time per request: 0.134 [ms] (mean, across all concurrent requests) Transfer rate: 1139.27 [Kbytes/sec] received

How exactly are you hung up with form data? Its low level, so you certainly can't get validation out of box, but otherwise in my experience getting form data out is fairly straight forward. I recently wrote my first Racket web app. Its not pretty or following best practices but you can take a look.

https://github.com/vishesh/whalebin

Thank you! Your project was helpful to skim through. Do you have any additions to the web-project-bootstrap.rkt project?

BTW, this is what I mean by form bindings being confusing: according to the docs, request-bindings is dangerous and shouldn't be used http://docs.racket-lang.org/web-server/http.html#%28mod-path... . That's kind of confusing, because request-bindings is used in the tutorial docs: http://docs.racket-lang.org/continue/#%28def._%28%28lib._web...

And the function that Matt Might uses, request-post-data/raw, isn't even in the docs! Haha, so handling forms has been a bit of a pain for me :)

Good to know! I followed the tutorial and they were using it there, and didn't see the docs after that. I see that the Matt's blog handles the problem with request-binding himself, by doing ensuring encoding conversions, and gets over the file uploading problem. So I would replace my code with the way he does.

For your boilerplate, you may want to add bunch of options to serve/servlet such static files path, stateless? (to not use continuations) etc...

The problems are not really Racket specific. They arise from trying to validate, parse, and execute upon the arbitrary input that can be passed in request bindings. The fact that all values are turned into lower case bit my as I worked on my first web-app. Getting bit by a cross site scripting [XSS] attack would have been worse. There's just a lot of work that goes into producing production quality webb apps when the app is worth producing that can't be covered in a tutorial. The Continue tutorial is a high level overview of the tools Racket provides (e.g. interfacing with persistant storage). The More tutorial gets deeper under the hood and covers dispatching on URI's with a server. Dispatching is generally going to provide simpler contexts for dealing with user data.

The request-post-data/raw function is in the docs: http://docs.racket-lang.org/web-server/http.html?q=post-data...

request is a struct with the field post-data/raw, so this implies that request-post-data/raw can access the field.

http://docs.racket-lang.org/web-server/http.html?q=request#%...

Just being exposed to Racket last month I was extremely surprised at how well it clicked with me as opposed to Haskel which I tried to learn for a few months on my own and just tried out SML and it moved to Racket. Can't put a finger on it but Racket just makes sense to me.

I felt the same way, Racket is ridiculously fun to use! :)

Have you tried python..? You might like it

I was trying to get exposure to functional programming. Yes I use Python (less and less) for many years, but I certainly wish there was a python when I was a teenagers.

Racket and Python aren't even remotely comparable as languages. Python is a far less sophisticated language with far richer libraries, and Racket is vice versa.

How fast is web-server/servlet? Is it fit for production use?

Last time I checked (which was, admittedly, a long time ago) it served noticeably fewer "hello, template world" requests per second than Flask running on Python 2.7 while using more RAM. This was a pity to me because I thought Racket could otherwise be a very interesting language for web development.

Isn't your `any?` function actually `ormap`[0]?

You should be able to do the same thing with:

    (ormap password-matches? line)

0 - http://docs.racket-lang.org/reference/pairs.html?q=ormap#%28...

Yep! Jay McCarthy pointed this out, and I changed it. :)

Very nice proof of concept. Just a word of warning: In practice you want to sanitize your input. The code in "Serving static files" allows to break out of document-root with "..".

Have you tried to actually exploit that? I don't currently use Racket myself, but I would expect request-uri to prevent ".." and if it didn't surely url-path would.

  ; extract the URI from the request:
  (define uri (request-uri req))
  
  ; extract the resource from the URI:
  (define resource 
    (map path/param-path (url-path uri)))
  
  ; find the file location:
  (define file (string-append
                document-root
                "/" 
                (string-join resource "/")))

Hijacking: Can anyone tell me how to make a Scribble website have CSS like in HTDP?

Extra CSS can be added when running Scribble:

http://docs.racket-lang.org/scribble/config-style.html?q=css

Reply on news.ycombinator.com