Tir: A Mongrel2+Lua Micro-Framework | Hacker News Reader

Tir: A Mongrel2+Lua Micro-Framework

sheddingbikes.com

120 pointshelium55 comments

Dear Zed Shaw, can you change the <title> tag of your posts to the actual post title and not the blog title? Always annoying to have to change it when I save your articles to Pinboard. But most of all, it's annoying because it's like, just wrong, dude! :D You can also tell me to go away and that's it's your blog and you keep it the way you want to. (That's fine too.)

Yeah I gotta work on that code some.

"The problem with Lisp is that it is acceptable to metaprogram until the only person who understands what you've created is you and The Flying Spaghetti monster."

Yeah, I guess this is sad but true. But as an alternative should I stop using macros and write lots of repetitive code?

The trick is to go far enough to improve readability, but not so far that the programmer has to remember all the metaprogramming or (worse) why you did it in the first place.

Speaking of "make things as simple as possible, but not simpler" what do you think of Flask?[1]

[1]http://flask.pocoo.org/

Hey hey, don't leave out Bottle[1], its inspiration.

[1]http://bottle.paws.de/

What exactly is your problem with the Arc code? Is it that the code that handles the form comes before the code that writes the submit button?

The problem is not that you have to remember the metaprogramming. I do not know what exactly `aform` expands into, but it works for me and I know what it is supposed to do. (Yay, I can make an argument from ignorance too)

I would still rather use Mongrel2 than Arc in production, but readability is not the problem.

Well it was mostly a joke, so you can loosen your booty a bit and have a laugh.

But, having worked with the average programmer and written a book for newbs I'd say the Arc code is just harder to use.

I find the Arc version much more readable, even though I have never used Arc. It pretty clear that (aform <x> <y>) is generating a form with <x> as the target page and <y> the form itself. (onlink <x> <y>) is a link with text <x> and target page <y>.

I think you got a WTF because you're thinking about a web application as a bunch of HTTP requests, instead of thinking about it at a higher level. For example you're thinking about a link like this:

1. When the user clicks <a href="url-of-the-target-page">blah</a>, he gets sent to url-of-the-target-page.

2. When the webserver sees a request to url-of-the-target-page, it does X.

Arc does not force you to think at the HTTP level:

1. When a user clicks a link (onlink "blah" X), he gets X.

Also, your code is not 23 lines. You are not counting the templates, which will probably double your code. Does your Tir code handle multiple requests? Does it handle the back button?

> I find the Arc version much more readable, even though I have never used Arc.

I find the intellectual dishonesty of fanbois irritating.

Surely people can have differing opinions without being intellectually dishonest, right?

A Roman and an Arab meet.

The Arab: Look how easy this is: 1234 + 4321 = 5555.

The Roman: I don't understand that. It's probably a trick, it only seems quick. How could anyone ever understand that? Look, this is understandable even though it is a little more characters: MCCXXXIV + MMMMCCCXXI = MMMMMDLV.

In what way am I being intellectually dishonest? That you (along with the 6 people that upvoted you) don't understand the Arc code doesn't mean that I don't. If anyone is being intellectually dishonest it's you. You are dismissing a piece of code that you don't even understand, and do not even try to understand. Then when somebody comes along and says that he finds the 4 line piece of code easier to read than your 40+ line piece of code, you claim he's lying.

FWIW, I have written a (toy) continuations based web framework in Ruby (the same technique Arc is based on). So when you still don't believe that I understand the Arc code and find it more readable, what can I say?

Open your mind and try to understand how continuation based web programming works. It's not that hard. Then you'll understand real reasons why it doesn't work well in practice and you'll be able to write some honest criticism of it (and as a bonus you'll understand why news.yc sometimes gives you "request timed out" errors). Heck, I'd claim it doesn't work at all in practice in it's current rendition for the vast majority of sites. Would you still call me a fanboi?

And you didn't answer my questions.

Indians invented the positional numbering system. Arabs only introduced it to your part of the world.

This is why tech guys are nothing without sales & marketing guys ;)

Indians (tech) -> Arabs/Jews[1] (sales & marketing) -> Romans/Europe (target market)

[1] Most of Aramaic and Arabic texts were translated into Latin by Jews in the medieval Spain

False dichotomy. It's possible to write readable and non-repetitive code. People do it every day without macro magic.

The Clojure community tends to be in favor of using core data structures and functions to compose, simple, reusable things, instead of using syntactic abstraction because it's there (like some people encourage). Clojure code uses macros, but most of the time it's to tie stuff together into a prettier, more convenient interface--sometimes to remove repetitive code of course.

So, why not use macros all the time? Not because they're hard to understand--you can always expand them to see what they do (or ask the original author to document it), but because they are hard to compose, and reuse. Sure, they can be used in special case a, b and c, but give me a higher order operators, and suddenly I can compose an opus.

Does this port to an imaginary arc-on-clojure framework look more readable to you?

    (arc/with-page-renderer
      (fn [content & options] (hiccup/html [:body content]))

      (arc/defop "/said" [request]
	(arc/aform
	 :form
	 [ [:input {:name "foo"}]
	   [:input {:type "submit"}] ]
	 :handler (fn [form-result]
		    (arc/onlink :text "click here"
				:handler (fn [_]
					   (str "you said: " (:foo form-result))))))))

I was writing functions like this (Clojure):

  (defn set-password [{"login" login "password" password}]
    ...)

Ok, Clojure de-structuring syntax already makes things pretty terse: the function receives and de-structures a map. But for functions with many args it was looking repetitive, so I wrote defn* which lets me write the same as:

  (defn* set-password [login password]
    ...)

So, do you think this is a fair use of macros? It has the disadvantage of hiding the fact that the function actually receives a map, of course (but I'm a solo programmer here, trying to save keystrokes).

This is, of course, a problem with any language. Start naming functions pr and arg and everything quickly becomes unreadable.

The other problem is that there is really no way for the reader to intuit that there is a continuation or that state is being captured and passed around. Using the word "lambda" in the onlink continuation might help in that respect.

Not storing the request state in the global environment would also help readability. One thing that shocks me about programmign is how few people realize that you can pass arguments to functions, and they can return values. Instead of:

   (omg-magic)

Why not:

   (send-response (handle-request req))

Hey, now we know that the parameter is a request and the result is a response. You wouldn't write:

   (let ((a 2) (b 2)) (add-and-print-a-and-b))

You'd write:

   (print (+ 2 2))

So why does this common sense go out the window when the problem is more complex than adding 2 and 2? I wish I knew...

There is no continuation in the arc example I think. The request is also not a global variable (the aform macro stores a closure in a global hashtable, maybe you mean that).

Depends on what you mean by continuation. There is no continuation as in Scheme's callcc. But there is a continuation in (onlink "foo" X). X is the continuation. The macro probably wraps a lambda around that, the resulting closure is the continuation.

Indeed, the first continuation is [onlink "click here" (pr "you said: " (arg _ "foo")) and the second continuation is (pr "you said: " (arg _ "foo")

    I needed something real because you cannot build
    a useful web server in a vacuum.

I’d even go as far as saying that you can’t build anything in a vacuum. This is my main gripe about many sample projects for web frameworks. You get the traditional and useless hello world and the blog engine in 20s. Yet none of them highlights how to decouple your app, how to deal with non mundane stuff (security, error handling and reporting). Most of us, when learning about a new framework, have many years of experience with other frameworks, so why not try to sell us on how your framework is different and better suited to the real world situations we’re facing.

I found flask's documentation quite well-written in this regard.

For example, they provide insights at how to grow your application's structure in several ways: http://flask.pocoo.org/docs/becomingbig/, http://flask.pocoo.org/docs/patterns/packages/ and http://flask.pocoo.org/docs/patterns/appfactories/.

Generally speaking, http://flask.pocoo.org/docs/patterns/ gives a good overview of things to explore in Flask.

Edit: I elaborated a bit more. Please note that I'm not advocating anything, just giving my opinion on their documentation.

Indeed, I’m going through the docs as we speak and I’m impressed by the quality of the docs and the examples provided are interesting. Thanks for reminding me about Flask.

you should definitely check out Pyramid then: http://docs.pylonshq.com/pyramid/dev/narr/introduction.html

This is true of many(most?) open-source library/framework/engine/API projects. They suck and don't do more than demos because they were written with insufficient perspective for anything beyond that. So do anything more, and they suddenly collapse.

Very interesting...

Made me jot down a non-mini, non-existent web framework I think I might like to use.... It's only just a tad bit more compact than Tir and will take a lot more effort to implement... but (I hope) it is much easier to tell at a glance what it is doing. Must find time for building my own web framework some day.

Corresponding Arc Challenge: (cheated a bit, if you want to make this 'proper' you'd need to add a cookie, and maybe another function.)

  require "filter"
  require "form"
  require "response"
  require "route"
  said_form = form.default { saying = form.charfield() }
  function said_page(args)
    return response.http(said_form.render())
  end
  function said_click(args)
    return response.http(
      ([[<a href="javascript:document.write(%s)">click here</a>]])
          :format(args.post.saying))
  end
  route.set(filter.get, "^/said/$", said_page)
  route.set(filter.post, "^/said/$", said_click)

  require "filter"
  require "template"
  require "form"
  require "widget"
  require "response"
  require "route"
  require "generic"
  require "css"
  
  login_form = form.default({ 
                  username = form.charfield(), 
                  password = form.charfield{widget = widget.password} 
                  }) 
              
  function login_form:clean(args, data)
      if authenticate(data.username, data.password) then
        return true
      end
      args:set_error(self, {'username or password is incorrect'})
      args:set_data(self, args.post)
      return false
  end
  
  function login_check(request)
      if request.session['user'] then
        return nil, response.redirect("/")
      end
      return filter.get(request)
  end
                              
  function login_page(args)
      return response.http(template.default.render_with {
          head = css.link("my.css"),
          body = login_form.render()
      })
  end
  
  function login_auth(args)
      if login_form:clean(args, args.post) then
        return response.redirect("/")
      end
      return login_page(args)
  end
  
  route.set(login_check, "^/login/$", login_page)
  route.set(filter.post, "^/login/$", login_auth)
  route.set(filter.any, "", generic.not_found)

Nice, yeah it's really not hard to crank these little frameworks out. In fact I'm starting to think, if doing one is so simple, then maybe just little frameworks from now on.

One thing though: Part of the point of Tir is that you can read the code to one function and know what it's doing. For example, if I showed this to someone and asked, "What happens after faile to login?" They'd have to trace through routing to figure it out.

Not that I'm saying that's bad, since the advantage is you don't have to save any state. I'm just saying that's what's different about Tir (and Arc or Seaside).

"NO TESTS. You must work at a startup."

LMAO. You have to hand it to Zed Shaw sometimes...

Now, seriously tho, this has been waiting to happen since forever. I always knew that when the right people started messing around with Lua, awesomeness would emerge.

Lua is the closest scripting language to C. It's faster than Python, Perl and Ruby[1]. It's not just faster, but also uses less memory. It's also extremely elegant (in language concepts[2], at least -- not too much of a fan of its syntax but whatever). But the fact is, apps running in Lua would be hard to beat in terms of single-node performance. Your $20 cloud box that struggles to keep hefty Ruby processes running (or even Python processes, though these are way smaller than Ruby processes) will suddenly be able to handle a lot more than it does today.

I think Lua for instance would make a lot of sense for the big ones: Google, Facebook, Twitter. Google has a lot of C++ code on its infrastructure[3]. Facebook resorted to C++, Java, Python, and Erlang[4]. Twitter took the Java path with Scala.

The more I build applications (and keep them running), the more I respect the Unix nature and the more I want to become proficient in C. Mastering a scripting language and knowing how to glue it to raw C, or something close to C, gives you a lot of power.

Imagine a Python+Lua stack, where you can have handlers written in Python and use small, specific Lua-based extensions to run complex computations. I'm already contemplating using something like Lunatic-Python[5] to accomplish that in an app where I have to create similarity-based clusters. Today the Python code that handles it takes about 15 minutes to run as a queued, background task. I'm very curious to see if I can bring that down with Lua-based code controlled by Python...

[1] http://shootout.alioth.debian.org/u32/which-language-is-best...

[2] http://www.lua.org/history.html

[3] http://www.quora.com/Ben-Maurer/Google-Infrastructure/answer...

[4] http://www.makeuseof.com/tag/facebook-work-nuts-bolts-techno...

[5] https://github.com/dmcooke/Lunatic-Python/blob/master/docs/l...

"Lua is the closest scripting language to C"

Hmm? Neither syntax nor semantics are really close to C, and considering the plethora of C and C-like script interpreters, this strikes me as a pretty odd statement. Or do you mean in regard to binding/FFI?

Binding, definitely.

It is also minimalist yet powerful (environments and metatables can do wonders), and very consistent.

If you're just doing Lua to get to C it's much easier to call to C directly from Python.

You can call C directly from Lua too, with alien (http://alien.luaforge.net/).

Generating the bindings really isn't that hard, though.

Before I read up on how Alien works, I have to guess: it uses dlsym() (or the Windows equivalent) to find a function by name, and has a number of prototypical function pointer types which it uses to call the symbol returned by dlsym(), based on the number and types of parameters passed in Lua. When calling C++ it could even check the parameter types, provided they know how name mangling works for whatever platform the run on.

Now to see how close I was...

Not to get to C, but simply to get somewhat better speed, /somewhere/ between Python and C, which is where Lua stands.

Strange, I haven't done a lot with Python but I've built a pretty big system in Lua and its FFI is incredibly nice, the nicest I've ever used, in fact.

Ah, I misunderstood. I thought he was intending to call C from Python by calling Lua from Python and then C from Lua. Still, calling C from Python is trivial with ctypes. Most people do not realize how easy it is:

    from ctypes import CDLL
    libc = CDLL("libc.so.6")
    libc.printf("boo")

It would surprise me if this was true because C interop and embeddability was part of the reason for the creation of Lua. I don't believe you could say the same was true for Python.

Wikipedia: "Lua is intended to be embedded into other applications, and accordingly it provides a robust, easy to use C API. The API is divided into two parts: the Lua core and the Lua auxiliary library."

Programming languages are about manipulating concepts. Give that Lua code to someone who doesn't have the necessary concepts (views, regexp-based routing, framework, dynamic websites, ...) and he'll be lost.

This rant just shows that the mighty Zed doesn't have the necessary concepts (and neither do I) to understand the Arc bit.

Edit: and obviously, reading the docs would help in this regard.

First, it's not a rant.

Second, programming languages are about making things. If I have to learn a ton of concepts before I can make things, then your language sucks.

Depends on how useful the concepts are. If, say, macros or continuations save you a boatload of work and make your stuff easier, then "you language sucks" is a bit of a stretch.

Obviously there are different trade offs of learning/doing, depending on your tolerance for it and how long your things take to make. For an afternoon framework maybe not so much, but for a longer project learning a bit more upfront makes sense if it'll save you time down the track.

It's actually OK if you need to learn a load of best practice before you're able to be productive. It stops people making a lot of mess.

The idea of process-per-interface is an interesting one. My first thought was that this has the "PHP problem" which is to say that if you have a bunch of useful utility classes and helper code you have to optimize those to minimize loading. In a framework like Django where an app is reused between requests, you can have a lot of useful framework available at your fingertips.

My second thought was more interesting than the first, though: Tir shows the value of Mongrel2 (or, at least, Mongrel2's approach). You can use the process-per-interface structure in places that make sense, and use Django-esque structures elsewhere.

Of course, Mongrel2 is not the only way to do it and there are sites like Amazon where they take the approach of packing the results of many requests to separate services into a single page, rather than having a single process be responsible for the page.

I find that these days the hardware's good enough that most apps that people create never make it to the "scaling problem" anyhow.

Good stuff, I think Lua might be an excellent choice for this. Now for some shameless self-promotion: a year ago I posted a Haskell solution to the Arc Challenge, which might be even more readable for non-programmers: http://news.ycombinator.com/item?id=1004701

Yes, but did it pass the ArcWTF Challenge. You should try it. It's actually pretty funny, but you also learn that maybe a bit more code is better for the brain.

I should try it, thanks. I don't think I agree with that you should optimize your readability for non-programmers, I think you should optimize (mostly) for programmers.

However, I know a guy who is a consultant, and in one project he built a small DSL to describe the domain of the client. Next, he sat down with them and walked them through it, and it was very easy for them to spot mistakes or logical flaws in the code. At first they thought he was crazy, but ultimately they loved it. So I do see your point, I'm just not sure if it's good in every situation.

Cool hack. I really like what I've seen of Lua so far. It would be great to see it become more viable for web work.

I think Zed's right that web applications are more and more constructed in terms of interfaces than pages.

Looks interesting. I found Lua to be a wonderfully predictable language, both regarding semantics and syntax. Which is quite nice, in an age of decorators, macros and weird histories (looking at you, JavaScript).

As opposed to Zed, I would say the same about Lisp, though (if you're a mature programmer, who went beyond the first excitement about macros).

Beating the ArcWTF challenge:

    page '/said' do
      form do
        message = input()
        submit do
          link("click me"){ text("You said #{message}") }
        end
      end
    end

Show this to a non coder and I'm pretty sure they'll recognize more than if you show them the 40+ lines of his Lua version.

Well the ArcWTF challenge was sort of a joke, but I think this code isn't better.

The big problem is you (and Arc and others like it) imply too much historical knowledge in the name of terseness. What happens is when someone has to fix the thing they have to dredge up all the knowledge of where things are rather than finding them right there.

For example: If I have to come and fix the template that has the input form in your code where do I go? Do I edit some code deep inside your framework? Is there a page somewhere? Is it in HTML? Oh it's not? So I have to hire a guy who is a designer and a Lisp or Java or Smalltalk guru? Ok there is a template, so where is it? Is that configured somewhere so I have to one more level of indirection to find the thing that configures the thing that shows the page?

If you back off the tersness a bit, you can then have a piece of code that's still small, but also complete by itself. Anyone coming to my code sample and told, "Fix the damn submit button so it's cornflower blue." Can do it no problem because it says right there what file is being used in what way.

Additionally, I don't buy it with these "yours is 40 lines haha" challenges. Who the hell cares? It's more about the usability and ease of maintenance.

There is no template. That is the code. All of it.

I do not think it is hard to follow at all. Definitely easier than your Lua code. It translates right into the description of what it's doing:

    On the page '/said' display the following:
      A form containing:
        an input box named message
        a submit button that goes to the following page when clicked:
          a link with text "click me", that goes to following page when clicked:
            a text saying "You said #{message}"

The problem you have with it is likely that it seems too simple. You're looking for additional code that isn't there. This way of coding with continuations as in Arc and in my snippet has many problems, including scalability, memory usage and bookmarking urls, but simplicity or understandability isn't one of them.

If it doesn't scale(for some definition of scaling - features, server load, etc.) and you have to rewrite it, it's still essentially hard to follow, because you will have to reconcept the entire system, and presumably stay data-compatible.

Zed simply advocates that you ignore toy systems if they don't give you an in towards writing a real app later.

I disagree. Something can be easy to understand even if it doesn't scale well. Bubblesort is easy to understand, even though it doesn't scale well. If Zed's claim was that it doesn't scale he should (and would) have written that, instead of writing that it's hard to read.

Reply on news.ycombinator.com