Is web application development supposed to be this hard?

I completely agree. Of course stuff like Active Record can save a lot of typing for back-end code. But: 1) a large system often requires hand-crafted SQL anyway. 2) hand-crafted SQL performs MUCH better for anything but the trivial stuff.

The biggest issue for me: FRONT-END development. Most web apps are pretty simple on the back-end, and coding database access by hand doesn't take long at all, I can live just fine without ActiveRecord.

Where I end up spending 90% of development time is on the front-end. Writing HTML code by hand (and visual tools don't help here) makes me feel like I'm in the dark ages: I miss one character and have to spend ages hunting down design issues. Javascript isn't much better for that matter.

IMO What's needed is server-side UI libraries, along the lines of windowing toolkits for desktop apps.

Reason I say server-side is because client-side libraries exist already (eg GWT) and most web apps still need SEO, perhaps unfortunately.

ASP.NET and Java Faces components don't really count either - you still have to write HTML code by hand to glue things together.

Ask any designer what a pain it is to get the layout correct in all the different browsers. How many CSS hacks are required? Just look how long it took Google engineers to make those Gmail buttons. Crazy.

Why can't we pull all these "best-practices" of laying out and rendering common components into a server-side UI framework, so a programmer can use his knowledge of the programming language of his choice to build a web application?

Moreover, encapsulating everything into components leads to re-usable code with proper inheritance (instead of html template "inheritance") - something every programmer appreciates.

This is what I'm working on now - a java library to do just that.

Sceptics will say this is a "one-size-fits-all" and it won't work. That's not what I've experienced so far. Sure, some web app may need a custom "component" - so you need to write one yourself. But from then on - it's re-usable.

Will all apps look the same, like on Windows? Although that may not be a bad thing, they won't - proper separation of concerns allows designers to style components to their liking with CSS.

The aim of a GUI framework like this is to make sure web apps stick to proper design guidelines, instead of a developer deciding that he'll make his own ajax scroll-bar cause it looks cool, though most visitors can't figure out what it is. There's a reason Apple apps look so professional. They are designed around human interface guidelines. Why re-invent the wheel for each web app?

I'm only doing this cause I got too frustrated with all the frameworks out there when trying to create my new web app. They don't solve my UI needs.

Who else feels the same way? I plan to open source the code, so if anyone wants to help, give me a shout.

> I don't believe the new Web 2.0 frameworks are the answer

I strongly disagree, there are a growing number of people that are discovering that abandoning the server for UI, liberates them from this atrocious development methodology that web developers have been locked in.

Using a prototype based scripting language (Javascript) for the UI and coupling that to strongly defended (security, type safety, validation) REST services gives you a rapid UI for users to touch feel and change while Strongly typed services give you the metrics that the IT department needs to insure they are not getting crap in their system.

This is why products like flash made huge inroads over swing and other UI development products because they offered the people that use them the speed and flexibility they need.

Java on the other-hand has been the enterprise standard for as long as COBOL has not. Because it is safe and rigid and has a huge library, all the things a UI designer does not need.

For two long we have made one party or the other suffer the sins of the other development methodologies practice. Even if you are a long gunman (I now am). It is still a far more productive way to develop web applications.

The client model become very clear when one abandon the server model. The funny part is it makes both sides so drop dead simple that anything beyond the toolkits seem like overkill. The frameworks are so light they almost seem non-existent. It is very natural once one liberates oneself of the years of indoctrination of doing it the old way and stop trying to just sprinkle some in.

One of the big Rails philosophies that could help any platform is DRY: don't repeat yourself! ActiveRecord et al attempt to give you the reusable building blocks that will be required in most sites. And in that regard, I think it really does reduce developer time. There are also drop-in basics for user authentication, etc. A well-designed extension should be mostly portable to other apps. For example, while I don't know of a SAML plugin off the top of my head, building it once could get you to 90% for future cases. The sophistication seems less daunting when you work iteratively, tackling small chunks of functionality.

Having worked on both the J2EE and Rails stacks, I find Rails to be way more productive. But that productivity can be costly: many things feel like magic if you don't understand what the framework is doing. Looking back, J2EE development seems to require too much ceremony for too little reward.

The goal for web development should be to spend time on the application's unique properties, rather than on the generic ones that everyone must build. A framework like Rails attempts to get every project on the same page for component interaction, but allows you to step outside the mold if necessary.

Having done both J2EE (at Sun) and Rails as well -- there is an order of magnitude increase in productivity. No question.

Many (not all) java web shops have difficulty putting out functionality at a rate that they need to in order to survive. Java is not the language for the web.

I have dabbled in all of the above and I noticed that I spend a lot of time up front lost in the framework. In fact the only framework that I have found useful out of the 4 you mentioned is Django. I think that is because it doesn't force you to use the framework shortcuts, they are there for more experienced users, but when starting out you can use more standard logic.

"Databases, web services, html code, javascript, business analysts, testing frameworks." yeah I never have to deal with any of those in Django....

I know what you're trying to say, but saying things like "Rails/Django" lack SAML is silly. It would be fairly easy to implement if needed, it's just not needed by the majority of us.

I've worked most of my career building web applications and while it isn't magic and does still take time, it doesn't take more time than I would expect usually. If it's taking 80-120 hours to put a simple app in production either you have a bad team or some seriously bad processes.

an analogy...

Man, writing poetry is hard. It takes me 2-3 weeks to write a haiku and months to write a sonnet.

Language doesn't matter for writing code any more than it does for writing poetry.

You need to specify a lot more details before discussing whether 80-120 hours is really fast or really slow for a given project. How many screens, 10 or 200? What kind of test coverage - none or 100% unit tested? How does it have to scale? What level of security? Do you have a style template or do you have to write one?

I work with java too. You can generate a simple crud app extremely quickly if you know what you're doing and which libraries to use.

true, but if you compare the same simple crud app to a desktop equivalent - which is easier? to get even the simple web app to the stage that it runs consistently on all the different browsers out there is a bigger task. say you want to list the rows from your db: do you use a table? or divs? lots of ways to layout these in html. then you need to write some CSS, cause there's no default look and feel provided out of the box. then you'll probably need to write some javascript to validate fields before user can submit a form (or alternatively come up with error-handling on the server-side and then write some html to display these appropriately).

so many different technologies involved. IMO a programmer shouldn't be expected to know java, css, xhtml, javascript - just to get a crud app running.

I've been developing web apps for many years and to this day some css layout issue takes me hours to solve to get IE6 to work properly. That's where time is wasted, not on the crud db access stuff...

Forget IE6, unless you have an client that demands it. I've only ever seen that once and it was years ago.

> java, css, xhtml, javascript

lol well that's the stack I've been working for 10 years so perhaps I am biased.

>that it runs consistently on all the different browsers out there is a bigger task.

This is not much of an issue anymore. jQuery and GWT do well enough.

>say you want to list the rows from your db: do you use a table?

Yes, you use a table. You're displaying tabular data. You should also be thinking about data models, not rows in a table.

>then you need to write some CSS, cause there's no default look and feel provided out of the box

Lookup some templates and read w3schools to make it functional then get a designer involved. That problem doesn't go away using some other server language.

I'm not saying it's impossible, it's just a lot harder than a desktop-app, because there isn't a GUI toolkit with standard look & feel, functionality, and API. Web designers ultimately write one from scratch every time they need to create a new web app. IMO that's wrong.

GWT is the only proper GUI toolkit I know, and it truly borrows lots from desktop APIs. But not everyone wants a 100% JS web page, some people still care about SEO.