Idea: using code rather than templates to generate HTML | Hacker News Reader

Idea: using code rather than templates to generate HTML

github.com

23 pointsjoshu66 comments

Answer to "why not just use code to generate HTML?": to facilitate communication between the programmer and the designer. If you work with designers who create the HTML for you, it is much easier for them to deal with back-and-forth tweaks and edits using a template. But if you are acting as both the programmer and the designer on a project, then yes it's possible that coding the HTML might be better.

s/possible/likely in my opinion.

I discussed this issue at http://softwaresimply.blogspot.com/2008/03/to-template-or-no....

It seems like people tend to prefer templates, however I'm not yet convinced. I think there are good arguments for language-generated code, especially with a language that is powerful enough. HSP in Haskell provides some interesting possibilities here (http://www.cs.chalmers.se/~d00nibro/hsp/thesis.pdf). Text.XHtml is another Haskell library that does this (http://www.haskell.org/ghc/dist/current/docs/libraries/xhtml...).

Not to rain on your parade, but this is a pretty ancient idea. I don't have links handy but I remember doing this with PHP (amusingly enough) around the turn of the century, and I'm pretty sure Perl has had something in place for eons. There are also a number of Python options if you google around.

Though, at least looking at the Python implementation it seems very clean and simple compared to others I've seen (though you're importing pprint twice), but then again I'm pretty opposed to the idea in the first place so I haven't really browsed the available options.

For a project 1.5 years ago, I generated all my html via markaby from _why. It really kept my html very very clean, but the downside was that the designer guy didn't understand what the heck was going on so I haven't considered using it since.

you've hit on why html (or css or javascript) generators will never catch on. they do not fit within the traditional workflow.

you cannot take a delivered html document and convert it into whatever html-generator you use. the designer will have no clue how to use your (hand) converted file in dreamweaver to make small changes. in the end, it just makes life harder for everyone and costs more money.

Your argument assumes workflows never change. If that were true, we would still be using whatever workflows our forebears used for every task. Since that is clearly untrue, workflows must be able to change.

However, given that business are formed to turn a profit, the standard workflow will remain unchanged until it is profitable to do otherwise.

If a markup-generating DSL was simple enough for designers to pick up quickly, it would make good business sense to push them through the transition.

> If a markup-generating DSL was simple enough for designers to pick up quickly, it would make good business sense to push them through the transition.

Designers being able to do it isn't the problem, justifying why designers should need to is the problem.

Ah, I was making a couple of assumptions:

  any DSL which generates markup must be more concise than the markup itself

Otherwise, there will have been no point in creating the DSL. This assumption is probably fair. An example of this would be Seaside's HTML generation DSL.

  a more concise language will be better in the long run

This assumption is flawed and doesn't take the designers' tools into account. A DSL would need its own tools before it became a valuable long-term investment.

Yeah, that's one of my objections.

HTML is a designers tool, and generating it is basically a programmer working by what they perceive as an annoyance by shooting someone else in the foot. It also smacks a bit of "I see no value in how this is done so I'm going to generate it from code", which I think is ignorant, but that's my opinion.

I also don't think it's as maintainable, but I have no evidence to back this up (but I really don't want to be the guy who has to go decipher the code that generates a website two years later, I'd even rather deal with ColdFusion).

HTML is a designers tool

I thought the modern idea was that CSS is the designer's tool, and HTML should only be used for marking up content in the most semantically "correct" way (which has nothing to do with design).

It can be a little tricky (containing elements and wrappers mainly) but I agree with this concept. It's just hard to put into practice.

Didn't expect there to be this much of a response this quickly. Some responses:

1) I want to do projects alone, without a designer in the loop. Why should I use a template when it's wrong for me? This lets me go faster. Therefore the workflow argument is useful, since this is hacker news and not designer news.

2) It's not a new idea. I have seen a bunch of similar libraries, although they tend to use datastructures, more than code directly. I think the small novel bit is that the object mutators return the underlying objects for further mutation (read the code; it's scary)

3) It's not a DSL. It is the language itself. The template programming languages are awful; as soon as you want to do something that they don't define, you are in hell. And the ones with embedded code are a nightmare to look at and format (delicious was in HTML::Mason.)

Markaby is pretty close to what I wanted to do. The other ones I saw for python were not so much.

4) I haven't seen anywhere the notion of context-aware encoding. Unfortunately, I managed to nuke that bit of source while cleaning up and renaming things before sending it to github.

5) I apologize for my code. I am still new to python.

Seaside (http://seaside.st/) does this. And the approach taken to an extreme works very well. It supports components with states with example.

Reading over the comments on this thread, it appears that you don't get to generate HTML from code unless you're Avi Bryant.

"I am Avi Bryant, your argument is invalid."

Seaside is for building web apps. The continuations make it possible to do the type of apps that would be impossibly difficult in most (maybe all) other web frameworks. On the other hand, it's not really optimized for your typical document-based website. In this case, the generated HTML is integral to the way the whole stack is put together. Even if you disagree with this decision (which indisputably raises the bar for designer participation), you can still respect the decision within the context of Seaside.

Replacing templates per se? Much harder sell...

I thought it helps to expand on this a little. (Feeling relived after 2 days of intense coding...)

Seaside is for building web applications, not web sites. I wrote about this difference sometime back at http://motionobj.com/blog/smalltalk-needs-a-framework-like-r....

But the approach of using code to generate HTML should be applicable for web applications, it just so happens that Seaside is not suitable, state-heavy for one.

The most common argument against code-generated HTML is this makes it hard to work with designers who work with HTML. But as a developer, how often do you change/tweak the HTML after a designer pass you a comp, either in HTML or a PSD? I always do, and I suspect it is the same for almost everyone else. Be it using templates or code-generation, the designer is not going to care, nor is he going to know how to structure the page so that elements can be shared or values substituted effectively. And if he tweaks the CSS, he can't change the HTML without working with the developer anyway, since that will break your code. So it's not like using templates makes the problem go away.

It really depends on team member skills and your individual process. If you have a designer who is not a web designer, but just a graphic designer who makes PS mockups, and he passes the comp directly to a developer, then that developer better be a competent web designer.

As an expert in both web design and programming, I can say that combining the needs of HTML structure for CSS and for template structure is a very subtle balancing act (nevermind the standardista bloggers with one-page sites who have 10-page manifestos on what constitutes a semantic class name). If you have a designer who knows HTML/CSS well, and a developer who knows the back-end well, they will need to have very good communication to arrive at anything approximating an optimal solution.

Now as developers we may intuitively feel more capable of understanding the nuances of CSS than a designer is capable of understanding code, but I think that does a disservice to the web design profession (eg. Eric Meyer knows more nuance of CSS in a very technical sense than almost any web developer ever will). The answer is, as always, "it depends on the individual." I have seen developers quickly get in over their head with CSS because you have to know so many arbitrary implementation and browser details to make it work right.

At the end of the day requiring a back-end to generate your HTML is a heavyweight process that excludes many talented people from the design/mockup process. Even as a developer, I often find it easier to test things out with a static document that run a big software stack and tweak database values to test output. Also, I find HTML wireframes to be much more agile than either paper or Illustrator wireframes. Even in the scenario you describe where the developer has to integrate changes from a designer into a template, I still find it cognitively easier and much less error-prone to see those changes with an HTML template vs markup generated by some other language. And what about when the designer needs to utilize generated markup as a basis for some design tweaks. In that case they need to probably do a lot of whitespace formatting and possible attribute reordering in order to get it to somewhat resemble the markup they originally created.

Of course there is a place for generated HTML, and I think Seaside really magnifies the gains to be found with the right approach, but again, it requires a very different approach from the typical designer/developer team.

How about something akin to X-expressions as often found in Lisp/Scheme. For example,

http://docs.plt-scheme.org/continue/index.html#(part._.Rende...

I've been sitting on this for a while and finally pushed it out, having never really had the chance to give the idea the work it deserves.

Anyway, the idea is simple: templates are a mess. They have goofy language variants, etc. Your editor can't color them. Why not just use code to generate the HTML?

Still needs: Context-sensitive escaping, css generation, javascript inclusion.

The reason why not is that eventually you will want to hand the site to a designer and not have to do a half-hour of work on it every coupla weeks for the rest of its life. Making things easier for the designer reduces your total work; forcing the designer to work inside the HTML your tool produces will either limit the nifty things she can do, or require you to come back and touch this code again and again. Even the tiniest templating system is more flexible.

(Sorry; pet peeve. ;))

Indeed, the most effective team will be at least minimally cross-disciplined. I think the desire to generate HTML via code is just a case of unfamiliarity. I've been writing code since 1986 and HTML since 1994, and I have to say that there is no advantage to generating via HTML per se (frameworks like Seaside notwithstanding). If you have a decent text editor it can color and edit templating languages just fine.

Developers should write HTML for the same reason that business people should learn to edit HTML--it's the best canonical format for all stakeholders.

What if you want to repeat a similar chunk of HTML in a loop? What if you want variables? Functions? <Insert general purpose programming language construct here>? The world has spent too much time trying to hammer programming language constructs into XML-based languages and templating systems. There simply has to be a better, way, but no one has successfully come up with one that satisfies everybody yet.

That's where template systems like Django's and stuff like PHP come in handy. I can pass either off to a designer who knows HTML and they are off to the races.

Certainly there could be a better way, but HTML and CSS are what we have (and frankly they work pretty well considering that the web comprises mostly documents). Emitting HTML from a programming language can be useful, but it doesn't solve the fundamental inelegance of nested markup. Solutions like this are just indirecting the problem, adding another layer of complexity without a meaningful abstraction.

...considering that the web comprises mostly documents...

I found this interesting. I've been doing web stuff since 1996, and I would have agreed with you back then. But these days my perception is that the majority of stuff on the web should be considered to be applications rather than documents.

Fair enough, though I would consider many many web apps to be document-oriented. It's a slippery definition to be sure.

My project, phpSprockets (http://code.google.com/p/phpsprockets/) does nested markup by working as a node system. After all, if you're going to generate HTML in code you might as well do it in an object oriented way.

$table = new Sprocket( 'table' );

$tr = $table->tr();

$tr->td( "OMG I'm in a sprocket" );

I've done similar things using XElement in C#, though using PHP's magic methods makes for very short readable code.

I agree that templates are handy for separation of concerns though.

If you've designed your markup well, this argument is a red herring; the designers aren't changing the markup, just the styling. It's also just not the case that templating schemes address this pain; any time your designer changes (bad) markup, you're likely to have to alter the code that produces it.

Well, I guess I wasn't smart enough to ever design my markup well. :) I actually think that if you believe markup can be designed to be perfectly flexible using the CSS we have currently (or had in 2004, the last time I actually had an original thought about this), it's probably the case that you haven't encountered a sufficiently imaginative designer.

Templates can be incredibly simple and require no further attention, but even slightly more complex ones like Smarty work really well, in my experience.

I'm not really even thinking about the complexity of the template language, as much as I am about the fact that all of the template languages shred your HTML document into tens of little fragments of HTML. There's always an impedence mismatch between the devs and the designers, no matter what you use.

I agree that virtually no designers will be able to work directly with something like Markaby; but then, nobody asks them to. They hand the designer the unstyled output of Markaby.

It's not well advertised, but Genshi is great at python based markup. It's mostly meant for inserting things into streams, but I could easily see it being adapter for full python markup.

http://genshi.edgewall.org/wiki/ApiDocs/genshi.builder

I don't want too sound negative, but there are a number of reasons this is a less than feasible idea.

First it's overkill. The reason we have a www at all is because HTML and its variants are dead simple. If the client decides that six key sentences should be yellow and emphasized, I go into my template and add a Yellow text! to my template.

Using your generator, I now have to figure out a whole new way of doing that.

The second point follows: Any first year web monkey can learn to add spans and em's. It takes a bit more effort to understand python. While this might not matter to someone who understands both, it offers much less flexibility to the already available alternative.

Django's template implementation is very, very good. It fits quite well into your average web shop and even fails silently.

I like your desire to scratch your own itch, but I felt I should answer your question.

If you ever name a class yellow, you've entirely missed the point of CSS.

I think it's safe to assume that the class name was chosen for demonstration purposes only.

Bad examples don't help to prove a point.

Yes, that's true, but I was trying to illustrate a simple concept.

Maybe it should be:

  <span class="client_wants_yellow_client_gets_yellow">Yellow!</span>

And if (when) the client changes their mind?

I'm pretty sure he was joking.

Er, or you could do Yellow Text!. But that's not even the best. What if he wants to change the color later? Special Text!.

CSS classes should be used sparingly and by purpose: they should describe the function of the element, and not the look of the element.

But that's just semantics ;) (har, har).

ASP.net uses code generated html for a lot of things: server controls, some user controls, and so forth. It can be a pain to override them if the markup they generate is crap (and from my experience, the built-in server controls and many of the 3rd party controls generate crap).

You could, but I get to use my yellow text elsewhere, sans emphasis. :P

Not calling it yellow I think is the more important piece. In fact class="sctmtdt" seems best to me.

* (Stupid Client Told Me To Do This) :-)

Advantages: Automatic tag closing. Near elimination of XSS opportunities (once you write proper escaping in the first place). Ability to take advantage of all normal code constructs without having to explicitly import them into the template language directly, which allows for some nice abstracting.

Disadvantages: Performance; it is intrinsically slower to process things on a tag-by-tag basis than to just blindly shovel out bytes from a template. Optimization may be able to get this to the point you don't care, but you're starting out behind. The classic "designers can't handle this directly" argument cited several other times. There are other reasons to think that maybe templates shouldn't be in a Turing-complete language (though I'd argue once you are there it might as well be a nice Turing-complete language and not a hacked-together one).

Personally, I'm not a huge fan of the "dumb designers" argument because any designer that dumb is probably lobbing in XSS vulnerabilities everywhere, but that depends on your framework. I also think that if your language starts out looking just like a slightly-respelled HTML that it's hardly any different than current template languages, which aren't actually HTML either. But most people seem to stop at the "designers" argument and consider it to trump all, no matter what the other advantages may be.

Personally, I lean in favor of using a code-based system because in my value system, "preventing XSS and other injection vulnerabilities" rates above everything else, even "designers can modify my code" and building HTML with code makes this much easier to enforce, but... evidence is pretty strong I'm unusual in this view.

One thing: use the "with" keyword so that you don't have to do td.close(), etc every time. It's cleaner and reduces bugs.

Also, I agree. I'm using templates that are just python functions in my new Python web framework, Webify (http://github.com/jperla/webify/tree/master).

I honestly don't think I've made it clean enough though (there are a lot of yield's there). Nevertheless, I think I'm on the right track.

The only thing I'm not sure about is that the Django people seem to think that allowing designers to edit code the html directly should be allowed, and that they need a simple secure straightforward html template language. Pure Python obviously avoids that. I also kind of like the straightjacket of Python templates.

I'll look at "with" -- I'm still finding my legs on python.

I like the idea, but it's not exactly new (at least not in the Ruby community).

The most known is probably Markaby, written by _why two years ago: http://redhanded.hobix.com/inspect/markabyForRails.html

Nokogiri also ships with a similar library, and Tagz is another one.

I've actually written such a library myself which uses ParseTree/RubyParser to parse the code and generates plain Ruby. Of course, nobody is ever going to use it, but it was a load of fun and it's super fast.

The python Nagare framework use both approaches of programmatic HTML and minimalist HTML templating. For example, to create a HTML list, in a div, from a Python list:

  from nagare.namespaces import xhtml

  l = ['the', 'mandatory', 'hello', 'world']

  h = xhtml.Renderer()
  with h.div(id='content'):
      with h.ul:
          h << [h.li(element) for element in l]

  print h.root.write_htmlstring()

Or a (X)HTML template can have nodes annotated with a 'meld:id=<id>' attribut. They are then retreived and manipulated in Python.

In both cases, one of the main goal is not to reinvent a new syntax / templating language but to use normal Python with all its power. Also the generated (X)HTML is always well-formed.

http://www.nagare.org/trac/wiki/PresentationTier

As per the python example, this has been done a lot in already, to name a few: http://www.hoboes.com/Mimsy/?ART=128 http://code.activestate.com/recipes/366000/ http://www.linuxjournal.com/article/2986

and others, as pointed out in Guido van Rossum's 1999 "Using Python for CGI programming"

If you have to communicate with designers, you have to use templates. And templates are only a mess when people insist in inserting $random_programming_language code in them.

If you have a language designed for templates from the ground up, like, for instance, Zope Page Templates, you can have designers editing valid HTML that's also a valid template, comfortably in their editors of choice.

And what if I'm the developer, designer, all at the same time? Why should I do something more painful?

For the day when you will not be able to be the developer _and_ the designer of the application?

Been there, threw away the t-shirt.

I have a day job. I want to build things better/faster/etc with my own time.

This is a good idea, but not the right approach. The key is to keep all the "design stuff" in HTML, so a designer can deal with it, and then express all the code as code. XSLT works that way, as does:

http://search.cpan.org/~jrockway/Template-Refine-0.02/

I like the second one better, for some reason.

I've implemented this in our Lua web application, and it makes writing HTML from code much better... although we still have templates. The API solves all sorts of problems, from unquoted attributes to unclosed tags, and all around makes life easier. I stole most of it straight from Seaside, of course.

I completely agree, may I suggest http://hotwax.infogami.com/HeRF as well? It's maybe not too fancy, but it's very short and simple to deal with. It also ensures correct tag closing without any special effort on your part.

             ["head", ["title", "HeRF Demo"]],
             ["body",
              ["div", {"id": "content"},
               ["h1", "HeRF - The HTML Rendering Framework"],
               ["hr"],

This seems almost as painful as writing out html.

Interesting you say that. I'm fleshing out a design that uses a similar syntax - In python:

(html, (body, (h1, "header one text") ) )

To me it's certainly not as painful as writing out html. I'm still working on it - hope to have a demo soon but its not a high priority, more something to sketch in the notebook when bored.

I have wondered if anyone would be interested in a templating syntax like this and got some response from the python mailing list -enough response to determine it's worth pursuing.

Its for personal projects so the designer seperation isn't a concern - I perfer to be able to generate it by code.

I think this sorta thing is more data-structure-ey than code-ey.

It's almost exactly like writing out HTML. The advantages are that you're writing something that your editor and your program can understand. The editor can highlight things appropriately, the program can insert or delete html easily since you've encoded it as a standard data structure.

Also, you may not have noticed that even though it can handle the data in tuples, if you write it in list form (as in your comment), it's trivial for a program to add extra items enclosed in a particular tag. So if I have:

   x = ["ul"]

I can just append elements to x to fill out the list:

   for i in range(10): x.append(["li", "Item %s" % i])

However, I don't think that it's _quite_ as painful as writing HTML... You don't have to worry about closing tags, for instance. That said, I really wish Python could deal with barewords in data structures, because that would really clean it up otherwise.

This is an old idea. Saw it years ago here: http://jakarta.apache.org/ecs/index.html

One example everyone's missing: the site you're on right now! The reason arc/HN can get away with using tables is that they're code-generated using lisp.

Paul Graham has been doing this since at least 1995.

Google web toolkit?

That's more about generating javascript+markup than solely markup.

Response.Write("<html>"); Response.Write("<body style=\"" + styleString + "\">"); ...

You all miss the point.

I've been thinking "templating system" for the last week none-stop. After seeing that project I just could not stop laughing. Yea why not code...

O well back to templating :)

Reply on news.ycombinator.com