Ask HN: Weby: A new python template language, can I have some feedback please? | Hacker News Reader

Ask HN: Weby: A new python template language, can I have some feedback please?

28 pointsljlolel36 comments

I have spent the last few years working on a new, minimalist, 0-magic Python web framework. It is still not done, but I have used the template language in production environments and it is a pleasure to work with. I want to release that separately today.

I have used Django Templates, Rails ERB, PSP, and many other template languages. Weby Templates have the advantages of simplicity and flexibility.

Blog and docs:

http://www.jperla.com/blog/2010/08/15/weby-templates-are-easier-faster-and-more-flexible

Repository:

http://github.com/jperla/weby-templates

View on HN

I really like the approach which is similar to the Clojure library called Hiccup. Generating HTML in this way is very productive if you're doing the layout and backend coding yourself. Traditional templating languages are a better fit when working with designers who will not want to read/write Python though.

However, I do feel the syntax is a little heavy compared to similar non-python libraries. It would be nice if it could be done with less characters - there is a lot of repeated 'p' and 'html' which pollutes the code. I have no idea if this can be improved upon in Python though.

Also, I'd recommend a big lovely README on the github page as people may end up there not having read the blog post.

I tried putting the docs on GitHub, but GitHub unfortunately does not have an HTML option for the README as far as I can see.

The markdown format should theoretically pass the HTML through, except that GitHub significantly modifies or ignores the tags so that the formatting is completely different. I will probably make a custom landing page a la Tornado.

Also, I'd like to add that even with the overhead of explicit p and html tags, this ends up being the same or fewer characters and fewer lines than traditional popular template systems including Jinja, django, and ERB

Hmm. Well. Most Python templating systems use HTML with some Python code intermixed, in one form or another. So you're basically writing HTML and adding Python for conditionals, loops, special function calls, etc. With this Weby library, however, you write Python code that generates HTML. I'm not saying that's bad, but it's a completely different approach.

Also -- and this isn't a criticism of Weby -- I can't help but notice the lack of Python's syntactic flexibility here. Don't get me wrong, I love Python (still), but this is definitely not an area where it shines. In languages like Ruby or Scheme (etc), similar code would have much less fluff. In Python, OTOH, such a thing wasn't even possible until fairly recently, when the 'with' statement was added.

Python does have limited syntactic flexibility. This makes it bad for making completely new languages. You are correct.

My philosophy recently has been that you do not want to create a new language. Haskell makes it easy to make a custom DSL. Unfortunately, that means a preponderance of new languages and mini-languages that you need to learn to get your work done. More importantly, these mini-languages invariably are missing features that you are used to in your powerful language: imports, decorators, debuggers, etc. Django templates did not even have variable assignments for years.

My recent philosophy is Pythonic: don't make DSLs but make domain-specific libraries. I have gone through many iterations to get this as minimal as possible while still being explicit and resisting putting in any magic. The results are still surprisingly readable after a little practice.

I have exactly the same stance on DSLs as you. Well said.

I'm coming around to the view that a DSL is a thing where you have actually created a new syntax. If there is no parser, there is no DSL in sight.

Few Haskell libraries actually provide DSLs, though I can name some that do, usually very small ones, often ones where if you still desire to write the raw Haskell you can. It's just that APIs take effort to learn and use, and Haskell can use some patterns that many other languages can't, just as Ruby can use some patterns many other languages can't.

I actually think seeing DSLs where there aren't actually any is somewhat dangerous, because people's brains stop being able to abstract correctly. I have a perl "DSL" at my workplace that I wrote that I have been desperately trying to convince people is not actually a DSL, because it makes people think about it wrong. It is a way of decorating functions in such a way that I can reach the metadata and do other helpful while they can get on with their lives, and it looks like this:

    special_sub subname =>
        permission_check { $_[0]->had('some_permission') },
        xml_rpc(name => "Category.subname"),
        does {
            blah blah blah
    };

and there are various other sundry things it can do easily that would make less sense out of context. This does things like expose this function to the local XMLRPC handler, and the xml_rpc function takes in the XML-RPC-specific metadata for this case. "xml_rpc", "permission_check" (which actually has a shorter name in real life), and even "does" are actually functions that return values which do the real magic, but I have the problem where people don't understand;

    my $common_permission = permission_check { $_->[0]->has('perm') };

    special_sub sub1 => $common_permission, does {...};
    special_sub sub2 => $common_permission, does {...};
    special_sub sub3 => $common_permission, does {...};

People see the DSL magic and suddenly its like everything they knew about the base language just goes flying out the window.

So you're basically writing HTML and adding Python for conditionals, loops, special function calls, etc.

With the maturing of the JS toolkits, I really don't know why people do this anymore. It is so much easier to write static HTML and overlay JS widgets for the dynamic parts. Then for search engines and older browsers drop a headless browser in front of it the website, so that no JS is required if the client cannot support it. To me the web templating frameworks represent a legacy mindset to developing web applications.

Clickables:

http://www.jperla.com/blog/2010/08/15/weby-templates-are-eas...

http://github.com/jperla/weby-templates

I built a variant of this style a couple years back http://code.activestate.com/recipes/440563-yet-another-reinv... . I used it to generate static content in advance, but it could have easily been used for dynamic content.

In my experience, when working as the only person on a project (all of the backend and frontend work), reducing context switch time in the form of removing languages can be a benefit. That can be augmented by knowing a templating language well enough (I personally got to that point with Cheetah and Spitfire), but then there is still javascript. Using Python syntax as html syntax is convenient as hell, but it isn't really all that maintainable once you go beyond the basics.

When your team moves from one person to multiple people, and you've actually got a frontend person who is really good at some templating language, productivity explodes, as the frontend person will be spending most of their time in that language. Also, you're not going to be finding someone with any experience in this new language you've written. The best new languages are ones that are evolving from previous languages, addressing the weakpoints in the former (django -> jinja, cheetah -> spitfire, etc.).

Finally, and this cannot be overstated, the insertion of business logic into templates is the worst cardinal sin of modern web development. Back in the wild-west days of the 90's through around 2002 or so, the use of ColdFusion, PHP, etc., was all understandable: we didn't know any better. But then we started getting usable templating languages in pretty much all programming languages, and they evolved. And the building of application logic into separate layers became not only possible, but became a time saver. Your backend guys build the logic to pass data into the template, your frontend guys format/handle that data into web pages. Business logic is in exactly one place. In the context of this language that you have written, while that isn't necessary, because your html templating is in Python, you will be tempted to introduce business logic into your template.

Otherwise, it is nifty :) Welcome to the world of templating.

I think you're over-emphasizing the importance of the separation of business-logic.

Essentially your templates shouldn't be making database calls, sure.

But just because your template language is written in Python doesn't mean your templates are going to be making database calls.

If the team is small and everyone knows Python well, I can see a pure-Python template system be far more advantageous than using a template language.

Template languages are simply for designers anyway who don't understand Python (and don't need to).

Good job on weby! Look forward to seeing the rest of the framework.

There is more to business logic than database calls.

And there is more to modern web development than your backend language and your templating language (hello javascript, jquery, yui, mootools, etc.).

Again, as I stated earlier, reducing the context switch time can help, and this may be low-hanging fruit for very small teams, but almost every Python programmer will be coming into this having used one of the big Python templating languages. And the moment you get a frontend person with usable jquery, yui, mootools, etc., experience, most of this "oh, try this new templating language that isn't like any other templating language you've probably ever used" is going right out the window; they are going to rewrite everything in the language of their choosing (it happened with our guy and django -> jinja, which was the right thing).

> Finally, and this cannot be overstated, the insertion of business logic into templates is the worst cardinal sin of modern web development.

Agreed. This is why I started using haml; it makes doing this _possible_, but really, really hard and ugly.

> p.raw(u'Hello, %s!' % thing)

yeah I'm not so sure I want to subject myself to Python string substitution for templates. Seems like I would make more mistakes and designers wouldn't be able to debug it.

<title>{{name}} | {{section}} | {{page}}</title>

p(html.title('%s | %s | %s' % (name, section, page)))

If I was going to obfuscate the way I write HTML I would use http://shpaml.webfactional.com/ but I don't mind HTML as long as I have a good editor like Eclipse.

Good point, would you prefer

    p.raw(u'Hello, ' + thing)

and

    p(html.title(name + ' | ' + section + ' | ' + page)))

or better yet (remember that these are all strings funcs)

    p(html.title(pipe_separate(name, section, page)))

none! The semantic HTML and template logic are both in python and thus look too similar in all of them. You want the HTML in the template to look like HTML and visually distinct from the template logic so you or other designers working on the project can style these semantic structures easily with stylesheets. Writing HTML with an IDE like eclipse that autotypes the closing of an element and autovalidates if elements are closed is easy enough, and there's no string-joining nonsense.

I'd vote against the name Weby because I don't want to live in a world where there is a web.py, web2py and weby/weby.py. Too much name similarity confusion.

Thats a good point. I also worry about confusion with spelling and the Webby awards.

The original name was Webify, but I switched to weby because 1. IBM owns the trademark, and 2. Weby is short.

This is all still in development, so if you can offer a better name then I'm all ears. The short name is important to me because I aim for this to be minimal.

Understood. Yeah I've also discovered it's hard to come up with a name which is both (a) cool, and (b) accurate, and (c) not already being used! Here's some random name brainstorming I'll donate:

YATL: Yet Another Template Language

YATS: Yet Another Template System

Webplates (meaning web+templates. though WebPlate.com & WebPlates.com both have a scummy adverlink squatter)

Minplates.py

HTemplates.py

PyperText (pyper = python + hypertext something)

PyperTML

PyTML (Python Text Markup Language?)

PyTML (Python Template Model Language?)

AntiTemplates.py

GenHTML.py

GenH.py

Ideally I can use the name for my upcoming web framework as well, a la Django Templates.

Also, the template language has an html helper lib which I describe in the docs, but it also has an xml lib, an rss lib, and really (since it's just strings) can generate any kind of textual template not just html markup.

IMO, the real killer with templates is form and input processing, so if you really want to show that this way is easier, you might want to emphasise some form processing stuff instead.

You also might be better off making your library importable as * , so your code looks more like this:

   from weby_templates.templates.lib.html import *
   ...
   with p(html()):
        with p(head()):
            p(title('Hello, World!'))
        with p(body()):
            p(h1('Hello, World!'))
            p(p('Please choose from the following items:'))

I know that Python normally advocates against import *, but it looks a little more readable (to my eye, anyway), rather than call html.blah over and over again. You also don't explain what that 'p' variable is in your example - some sort of request object? It's also not clear why you need to call it to generate HTML.

Another thought: what if you used nested functions/classes instead of the with statements?

    html(
        head( title('Hello World!'), ... )
        body( h1('Hello World!'),
            p('Please choose...'), 
            ul( li('Option 1'),
                li('Option 2'),
            )
            ...
    )

With the html() function returning the relevant HTML. That way you get your string-returning and automatic tag closing for 'free'. I think there are a few lisp/scheme libraries which use this sort of style.

First, I'm really glad you took a look at this and thought deeply about how to improve it.

It is possible to do import star, and if you really know what you're doing, you can. But, you quickly start running into issues with polluting the namespace (which is why Guido says not to do it). For example, one HTML tag is p, so now we have to use a different accumulator. I find p very natural because I think "print" every time, I used 't' for a while (months) but it just never clicked. Furthermore, HTML has other tags like a, so now you can't use either p or a as one-off variables anywhere without weird bugs or confusing reads of the code, not to mention other common names.

Moreover, I even tried using import html as h to save typing. It actually made the templates less readable (I think because of less long explicitness). It can take a few minutes, but actually you quickly get used to the repeated p(html( and it just becomes a useful guidepost when reading through a template quickly.

Your second suggestion of a language is nice and clean, but you lose a lot in terms of flexibility and power. For example, you cannot easily loop, or insert a readable conditional, and you definitely cannot assign a helper variable which even Django now acknowledges can be useful.

I have been using these templates in production on medium-sized, complex sites for years. I have added enough functionality and simplicity so that I can do everything I need to do, plus it is obvious how to do it. A lot of toy template languages have not seen extensive use in a variety of applications with heavy load, so they are missing subtle, key elements which can be show-stopper issues in practice. Finally, languages keep snowballing massive APIs (to patch up these issues) which become so difficult to completely learn that easy things become hard.

> I find p very natural because I think "print" every time ... I can do everything I need to do, plus it is obvious how to do it

Probably more because you've used it for a while. I generally prefer longer variable names to help with readability, which makes the accumulator/with technique more verbose and harder to read.

If you don't like import * , you can use a combination of from ...lib.html import p, h1, h2 and from ...lib import html, but as long as your namespace is clean (ie. use __all__) and you're clear that the module is designed to be imported that way, I don't see that it's a huge issue, particularly if it clears up a lot of clutter.

There's also nothing in that API that prevents you looping, or assigning helper variables - you just store local variables before you generate your template or create sections of your template a bit at a time. Or use list/generator comprehensions, which would fit the functional paradigm a bit better. A couple of examples:

    ul( *[li(option.name) for option in options], style="dropdown", ...)

    def index_page(request):
        my_name = reqest.GET.get('name', 'Bob')
        head = html.head( title('Hello, %s' % my_name))
        body = html.body( h1('...'), p('..'))
        return html.html(head, body)

Of course, all of this is developed using the POOMA methodology, so there might be some hideous limitation, but I can't see any from a few minutes thought.

import * is the devil, really don't use it. Instead, hack up this recipe: http://code.activestate.com/recipes/277940-decorator-for-bin... to inject the common tags into the local namespace of the function. It will have the side-effect of making the code execute faster.

As for the callable variant, there are a few versions of that (nevow.stan was the first one in the style, I wrote a version a few years ago that I referenced in another post here, and I've seen a few pop up since) While it does work, indentation can get funky, looping gets tricky (you end up looping quite a bit with data-driven templates, it turns out), and closing parens aren't nearly as useful as a dedent. Being able to mix both styles may be a win.

Great comment! Agreed. I'm a little afraid of that hack though and what it might do to a query optimizer.

You're right that the callable system makes loops tricky. Also, great point about the closing parens. We use python specifically to avoid them and to just use whitespace!

In practice, I have done a mixed approach as you suggested. I used this iterative style with limited callable style embedded where that makes sense (everything just returns a string, so they embed cleanly).

Jesus, that's nasty. Hacking python bytecode is far, far worse than import *. I'd be more inclined to just use __all__ in your module to hide the bits that you don't want.

Could you reduce the noise by having the accumulator (W in the below) support HTML generation methods?

    @weby.template()
    def index(W):
        with W.html():
            with W.head():
                W.title('Hello, world')
            with W.body():
                W.h1('Hello, world')
                W.p('Lorem ipsum dolor sit amet')
                with W.ul():
                    W.li('Lorem')
                    W.li('Ipsum')
                    W.li('Dolor')
                with W.div(className='footer'):
                    W.p('foo bar baz bim')

A smaller win is to use keyword arguments (instead of passing a dict).

Those are good ideas, and I experimented with those. I also appreciate that you actually looked into it and thought deeply about how this can be improved.

You gain a handful of characters you don't have to type.

You lose a lot of flexibility, however. For example, what if you want to use 2 different libraries in a template? Now, you cannot do that since the W accumulator is bound to one of the libraries, or if you do bind multiple ones then they may conflict in odd ways.

But maybe that's overstating the problem, worrying too much. Maybe it's okay that within each function we just assert that it can only use one library. I'll think about this more. I'm still a little worried it might be hard to explain. Do you think it'll be easy enough to describe to someone that W is both an accumulator and a library in one?

The dict idea would save a few characters, but it leads to problems with, for example, attributes that conflict with other arguments in the function call, or Python keywords. Even in your own example, you put className instead of class because class is a restricted keyword. It is important for me not to have to force the user to have to learn (and remember) that convention. Moreover, in this case of the class attribute, the most common kind actually, it saves no characters.

I like the general structure of what you have created but I will offer up one small thought. I grabbed the sample code from your blog page and did a rough character count excluding indenting. You have 444 characters of template to generate 270 characters of output HTML. I know this generalization of course doesn't encompass all the other advantages of templating documents but I would give a thought to improving the compactness of the syntax.

Please note that the docs are completely static. Because they are static, I could have generated these docs by just writing a function that returned the whole html string, which would be a small constant overhead. The only reason I used Weby Templates is to show off the features.

In a normal programmatic setting, you would have significant if/else constructs and looping. These templates would be significantly shorter than hard-coded HTML, and still shorter than the popular Jinja, Django, and ERB templates because

1. they close tags for you in a Pythonic way. 2. it is trivial to create functions and filters to do repetitive tasks for you, whereas they can be difficult, hard to locate, or a pain to do with the others.

How is your templating solution different from these?

http://breve.twisty-industries.com/

http://code.google.com/p/dirty/

genshi.builder is similar too.

More here

http://wiki.python.org/moin/Templating

I don't care for it. Jinja is pretty much the last word in Python templating, IMO.

I think this is an unfair comment. Weby templates are a different approach to Jinja (which is a conventional templating language) and its like comparing apples and oranges.

Automatically generating HTML from code has many advantages (and some huge disadvantages) compared conventional templating but has been used as far back as Perl's CGI.pm with many different lisp libraries also doing the same.

If you have never used such a library I'd recommend giving it a try sometime - they're very nice in situations such as automatically generating forms or database search results (where conventional templating gets challenged) which have more display logic than HTML.

I don't think it's comparing apples to oranges. While they take different approaches, they have the same end result, and in a given project, you would probably only use one.

Apples and oranges are just different approaches to getting the nutrients one needs. It seems like an apt comparison. Whether the normal usage of "apples and oranges" is sensible is a different matter.

I'd appreciate it if you tried and use it a bit and then give me feedback on what is difficult to learn because it is overly esoteric or specifically what you could not do.

Some advantages are that it integrates more cleanly into Python code, the templates end up being smaller, you get guaranteed closed tags, and it has a significantly smaller API and thus it is easier to learn while being quite powerful (very Pythonic). It does all this without compromising on speed, legibility, or power.

Reply on news.ycombinator.com