Ask HN: Why Isn't the Construction of CRUD Web UIs 90% Automated? | Hacker News Reader

Ask HN: Why Isn't the Construction of CRUD Web UIs 90% Automated?

201 pointsdavidjnelson186 comments

It seems silly to write the same types of code over and over to build CRUD web user interfaces. There have of course been hundreds of UI builders. Why aren't they used? Styling should just use plain CSS. Behavior should be mostly auto generated config from a standardized format such as json schema or a db schema. Behavior customizability should come from tweaking the auto generated json config, and adding some HTML and JavaScript to higher order components. The build targets should use plugins, starting with angular and react. Components should use a shared plugin infrastructure. Designers should be able to prototype in an invision like tool with real CSS and HTML that can be checked into source control. Why isn't this the way things are done?

View on HN

I suppose a better question is "Why aren't you using a framework that automates 90% of your CRUD?"

There are lots of those. Rails is the obvious one to point to, but there are dozens of others built in the late 00's that all tried to recreate that "20 minute DB-Backed Blog" demo.

I imagine the answer (to my question, and thus yours) is that fashion changed. Everybody wants to build Single Page Apps today. For everything. Even when it makes no sense whatsoever to do so.

So you find yourself dragging in a few megabytes of javascript just to render your static blog page. And you have all the power that made sense for the engineers at Facebook to use on their newsfeed, where things would regularly change behind the scenes and the page needed to re-render and re-flow itself several times a minute without the user noticing, for which a brief 3 second "flash of empty container" was a reasonable trade-off. And you use none of that power on your page, but you have it there in reserve, just in case. Because that's what The Trend has told you to do.

Step back and build some pieces of your site with tools that can look at your database and spit out some sensible defaults for CRUD screens, shamefully pre-rendered on the server as they may be, for the pieces where it makes sense to do so.

Fully agree with you. If you use Ruby on Rails (but any modern full-stack framework would do the trick), building a CRUD is indeed 90% automated already.

All you have to do, and that's where the value lies, is putting the bricks together and building the other custom 10%.

Modern full-stack web framework have pushed productivty to an unprecedented level.

Rails handles 90% of your needs out of the box. As a developer your job is to add the other 90% of the functionality the client wants.

How does rails help you automate 90% of the creation of the angular or react single page application ui?

Tell me about it. I'm currently working in a javascript shop where we recently finished a 4 month react project with 5-6 full time developers. The website had about 10 pages, with about half a dozen forms, all but one of which were very straightforward, and not a ton of fancy UI.

As an experienced Rails developer, I could have built the whole thing myself in about a month and a half. Less if the html was provided. There was absolutely no reason to build this thing in React. But, the architect and lead dev have decided we're a javascript shop, damn the torpedoes. For certain things we do it makes some sense, but mostly it just slows things down and makes development unpleasant.

Currently working in a Wordpress shop. I feel your pain. Because the client might want to blog in the future, so we build everything in Wordpress with this one theme based in ZURB Foundation developed by one guy that incorporates its own build system in Gulp for Sass that breaks the Wordpress theme engine if you don't have the files in the right places.

Ugh.

I hear you. Even back in 1998, I could bang this stuff out immensely fast with perl, cgi.pm, and a few libraries I created for reuse.

The thing is, the user experience of these Web 1.0 apps is absolutely subpar to that of a modern single page application.

What’s ironic is that Rails’ turbo links improves CRUD performance almost to the point of an SPA with close to zero impact on complexity.

Unnecessarily building websites as SPAs, while likely assuring employment for many, is a suffocating experience, especially in the long run. I just can’t make myself do it.

I absolutely agree with this. With turbolinks I get the fast page load feel of SPA with much less code and a far superior and maintainable architecture.

Other than 37signals, which large tech companies do this? Large tech companies have smart engineers. If turbolinks really were better, why haven't they caught on?

I sometimes suspect it's because most web development is boring and to make it interesting smart devs make tools and frameworks instead, even when it's not really appropiate.

Is the Facebook interface today significantly better than it was 5 years ago? No. But they've invested a huge amount of time in React in that time, so why isn't it?

I'm going to play devil's advocate here. I totally hear you re: everything being built in React nowadays and SPAs are annoying both as a user and a developer, I definitely feel like that sometimes.

But the answer to your question, why isn't Facebook significantly better than it was 5 years ago, is exactly because they've invested a huge amount of time in React etc. and now utilize them fully to build many more features on top of it.

5 years ago, your notification bubble would often go out of sync. Your inbox chat might be on a separate page (can't remember if they had the bottom chat bar in 2012 yet). Even if it was, it definitely did not have video chat and voice call straight from that. Notifications didn't appear at the lower left in addition to the notif dropdown. I could go on forever.

It is very similar to mobile phones cpu/memory and battery. Battery tech is getting better and better, why do phones in 2017 still only last < 24 hours, instead of 3 days? It's because battery got better, all the phone manufacturers (Apple, etc.) made use of that extra power and put better cpu/memory in their phones. Now if phones have better and better cpu, why are newer phones in 2017 still feel as sluggish as the ones in 2007? Again, it's because the phones have better cpu now, mobile OS and software developers put more and more features in their code and make use of all that extra cpu resources.

Users won't be happy these days with user interfaces which are not single page applications unfortunately. So we need "rails for react", or "rails for angular".

The truth is that the construction of CRUD web UIs _is_ 90% automated.

The web and storage frameworks do the vast majority of the heavy lifting - the application code is really just simple wiring and validation.

The trouble is that (in my experience, at least) many web shops and web developers enormously overcomplicate that application code; shoehorning enterprise "layering" into simple components, creating poor - and often unnecessary - abstraction. These things lead to grossly over-engineered solutions with bloated code and control flow flying all over the place.

So the reality is that what can be automated largely is already - we are just, by and large, pretty bad at doing the bit that isn't.

One can compare it to any other product in the real world. Just think about cars. Every car is about the same nowadays, many uses the same parts under the hood. But still, every car has different functions, different designs and slightly different placement of things UI-wise.

This is the state of the development world as well. It is like this because not everyone wants the same generic model and every company has to stick out in some way.

The thing about cars, or any real-world comparison, is that software is always changeable. Its hard to change manufacturing lines over a new design for cars, thats why they have designs out there for years at a time.

I think the vast majority of our time isn't wiring a UI to CRUD actions, its dealing with the complexities of whatever business domain the CRUD app is intended.

It's also worth pointing out that the design cost of a new model of car (which is equivalent to a new VERSION of an existing software product) is at least 1 billion dollars.

If a really good big crud app costs 100k on upwork is more similar to remodelling an office. In many ways.

http://bramaindustries.com/wp-content/uploads/2015/05/office...

Software is changeable but I think it's hard to change stuff for large applications as well. Just think of the massive work that goes into redesigning an enterprise application that users have been using for years.

It's not that easy..

> not everyone wants the same generic model and every company has to stick out in some way

You also have all those little edge cases that do not fit in a generic CRUD model.

My experience has been that shops that avoid abstraction (namely, lacking a strong separation between controller code and business logic) end up suffering a decent amount long-term.

There's an argument that many layers of abstraction lead to too much boilerplate, but lacking abstractions can cause slowdown long-term because it gets harder and harder to handle all the interactions between subsystems.

You can always automate boilerplate "generation". You can't automate inconsistent business logic.

The problem is the idea that "layers" and "abstraction" come as a package. You don't need layers to create meaningful and useful abstraction - in fact more often than not they prohibit it.

And you certainly don't need layering within a single codebase - if I see one more codebase where everything is a FooController, FooService or FooRepository I just might go insane.

People abstract logic away from controllers only to dump it all into a massive procedural "service" class, which leads to no end of pain down the road. Once you stop seeing layers and start seeing a vertical dependency tree starting at the controller, the world gets a lot more rosy. Consumer-driven development helps here.

layers of abstraction may help agencies that move between different projects for different companies in first of all being able to reuse what they have done between their customers, but also in building something that someone else will find difficult to maintain - thereby providing lock-in.

Interesting, this is how most of our code ends up . But the service abstraction has been beneficial for our other applications which consume them too. I’m going to read closely about Comsumer driven approach though.

This is a very good question because it gets into a deeper issue that I find in all corners of real world programming and results in large complicated applications even when they use powerful frameworks.

Real-world programming is absolutely choc full of things that look like simple applications of a well known solutions, but which turn out to need lots of special-snowflake code to actually implement. Part of this is unforced errors, but usually I can't look at other people's work and say in detail how I would have done it substantially better.

So I think there is something fundamental going on. One guess is that although we can write tools that automate the perfectly common case, we are not good at writing things that smooth the inevitable app-specific corner case.

I agree. I think its best to think about it in terms of high dimensional spaces and manifolds. The manifold of solutions to business problems is currently not well approximated by the way that we decompose business problems. And by this I mean that if you were to look at the manifold of problems solvable using current techniques in amount of time x, you would find that in general that manifold is a long way away from the manifold of useful business applications. I'm not saying that there is a better decomposition - a better decomposition that leads to a manifold that lies closer to the target manifold - it may be that the manifold is complex in a way that makes it impossible to approximate efficiently in terms of time to develop.

And we may be butting up against the fact that most objects in mathematical space do not have effecient ways of representing them. I'm thinking of kolmorogrov complexity and the pigeon hole principle. Most outputs have no short programs to produce them.

Being John Manifold.

Although a bit subtle, I think the line of exchange from top and down to here is amongst the best I've seen.

100% this - I've spent many years feeling like I'm one abstraction away from having a killer library that would get me the initial crud based app done in a single line of code...

I've never actually managed it - the edge cases mean I end up chasing my tail around instead of "just coding the app".

Now I use small libraries for individual parts (like menus and forms) but even then 50% of the app becomes special edge case forms anyway. Accepting that the job is to just implement the spec and not invent a rube goldberg machine is an important lesson to absorb.

The goal is to have something that makes "just coding the app" simpler and faster without sacrificing 100% customizability or user experience quality.

I think even with edge-cases, the question remains valid why you can't automate the common cases.

I wonder how often it's actually non-standard requirements and how often it's a mix of processes and poor communication/changing requirements.

I think in many projects, in hindsight it's perfectly obvious it is just a standard CRUD app - but it might need a few iterations to get the understanding of all stakeholders to that point...

Also, seems to me that such tools clash with the organisational process of how webapps are created today: It almost seems to be required that a web app has some kind of custom-written "business layer" in between, even if that business layer doesn't do anything than pass requests around.

I think heavyweight frameworks often make the edge cases harder, because when you need to do something "unusual" you end up fighting the framework, and now you end up with a complex program where the framework as well as the buisness needs count as the problem domain.

But you are likely onto something that some things are just standard apps even if you can't see that at the time. And my mind might not be well tuned to seeing it even in hindsight.

Which makes me think that banging out simple apps fast is a very specific skill, because you need to see how to meet business needs within the slot provided to your by framework.

I think it mostly comes down to user experience. The best user experience is usually going to be specifically tailored to an individual application. It won't be very generalizable.

Apps created with ui builders (or even with just a constrained set of ui components like bootstrap) tend not to have good ux since every workflow has to be shoehorned into an existing abstraction. It's convenient for the programmer (and the budget), but not for the user.

A really good example of this could be validation.

Almost every CRUD app builder includes the ability to validate fields and properties, and this would appear to be sufficient for nearly all scenarios except it is not.

I can not name a single CRUD app that I have built that didn't require multi-field validation... i.e. start date prior to end date, time must be completed if date is filled in, if this part of the form is filled in then that part must be blank.

Without this capability to really express the application business logic, the technical simplification fails short.

The reason for that is because we tend to "stop thinking after automating CRUD". I think CRUD is only the basic tools on top of which "Actions" and "Streams" need to be built.

Thinking about scenarios, example "Sign up" is not really just "Create a user row in db" but probably assign them a group also, and more.

Similarly, we rarely would use a "R" in CRUD (except administrators). Read is a basic operation, more frequently we need transformations/filters/aggregates on this Read, hence streams.

I have lately been trying to formulate the 90% of the 10% remaining. Checkout my other comment if interested.

The existence of multi-field validation is a sign that your app can't really get a way with an "anemic" domain model -- i.e. the app is not just a front-end to the DB.

What's really going on here is that you have a domain object with some nontrivial invariant. It's a mistake for the UI or anything that is aware of the UI library to try to model these domain invariants. Instead a call should go the model layer which then returns (or throws) an error providing enough info for the UI to gracefully help the user.

The UI framework can play an important role in this: but that role is to help programmers provide a UI for the validator that lives in a model layer. It is not to do the validation itself. (Although frameworks should provoide an arsenal of simple, single field validators, of the "this field has data-type phone-number" kind).

Sure, I don't disagree.

But for the current state of CRUD application frameworks that do not do this, it implies some enforcement at the data layer and probably some hint at this at the ORM level. This really does not exist in any sufficient way.

It reminds me of the idea that a graph/diagram generator that used some generic data source could be a common analytics tool, and that with a few well-defined defaults no-one would need to run lots of different analytics tools, or they could be generated automatically and would be good enough.

In reality they never are good enough. There is no one-size-fits-all analytics system because the KPIs differ from company to company, project to project, and these differences go all the way down.

Sometimes we see something being sold as a "this is all you need", i.e. a SAP solution or some generic tooling, even Stripe, etc. But the level of customisation typically required to make it sufficient for a mature organisation to use it is typically quite high.

A strategy to resolve this usually is: Change the org to fit the tech, rather than the tech to fit the org. But I've seen this fail consistently as typically the tech is too simplistic.

Then there's the "customise it to fit your needs", but going back to custom multi-field validation, or analytics... the number of permutations of logic is immense and the customisation tools themselves only allow simple expression of that customisation.

The end conclusion is that we need extreme levels of customisation, but when provided through tooling the effort to perform and manage the customisation (with auditing of who changed what, when, etc) is such that actually it is simpler and cheaper to write the customisation in code.

We absolutely should write the customization in code.

Have you considered that maybe CRUD application building is actually already 90% automated, and the things you discuss are the remaining 10%?

Hah, good point. Be thankful we don't need to rewrite an ACID data store everytime we need a crud app

Count how many datastores you can use right now. I bet we could find 200 active data stores.

You make a great point, but we can and will certainly do better as an industry in the future.

The most famous article on this topic is called "No Silver Bullet". It basically answers this question: split up the work you need to do into "essential complexity", which is specifying in minute detail exactly what you want your app to do, and "accidental complexity", which is all the programming work around that to get it done. The "essential complexity" is still most of the work.

Thought experiment: If I gave you a programming environment in which you just said, out loud, exactly what you wanted to have happen, and there was no programming, it just happened, how fast could you build a CRUD app? Significantly faster, true, but there'd still be tons you need to specify, just because you want something that does something specific.

If you want an example of something that that is fully automated, think of a Facebook page. You can build it in seconds, but you're sacrificing tons of customizability. In terms of our thought experiment, this is like saying "make my app exactly like that". It's very efficient, but lacks most customizability.

There are many things that we do or could do to make the ssential complexity more manageable.

Something that we do already is abstract away parts of the essential complexity (CRUD, entities, validations, etc) where you have DSLs to specify that essential complexity.

But there are many other things we could as well, like some sort of ontological autocomplete that would help you specify your business model (a User may have these fields, a “Sale” is better represented as a “Transaction” in order to handle returns)

Moreover, re the Facebook App example, the ease of creation vs control is a false dichotomy. A page builder of sorts could spit a predefined design (to have something working ASAP) but there is no reason you couldn’t customize completely to your liking, as if no page builder was used. In other words, having a working app ASAP by using defaults, in order to aid requirement discovery, should be a feature or programming environments/tools/libraries framewors and not an either-or.

I'm not sure I agree.

The essential complexity usually isn't that hard.

From a high level view - most apps are basic.

Even a second order articulation of function i.e. 'screen here, buttons here, images here, button does this' etc. is not that hard.

Below that, it's kind of a mess.

The byzantine tools we have (I'm talking to you HTML/CSS/JS), to complete the task engender a massive amount of 'arbitrary complexity' and make doing anything really quite hard.

Which is why, by the way - once you have a Dev that becomes 'really familiar' with a 'specific stack' - they can move really quite quickly.

But most of the tools we have are a mess, and they keep changing, hence, all the complexity in our systems - which grows exponentially across more and more developers until you have any number of very smart people all trying to paint a Picasso at the same time.

FYI - I believe that making an app shouldn't be that-that much more complicated than making an FB page.

There isn't really a lot of 'essential complexity' in most apps, I think.

I agree to some extent, but I wonder if you actually tackled a web project of more-than-trivial complexity?

I did, and tried to find a number of good abstractions (for example, generating UI / ORM according to data structures), but in the end didn't come up with something satisfying. It's too easy to paint yourself into a corner.

In my opinion one of the big sources of essential complexity is the layering. You have representations of your data in the DOM, in Javascript objects, in HTTP bodies / URLs, in server-side plain text representations, in the server-side language's objects, in the database... And strictly speaking you have to make sure (de)serialization works and data integrity holds at each of these boundaries. Also, many functional parts need to be present in more than one of these domains.

I've worked on big and small.

Even for the 'big ones' the (there were network management apps) - the 'business logic' (networking logic in this case) - was not rocket science in the end.

As you indicated, not only is the 'layering' a pain, but often there's a whole set of technologies and toolchains at each layer!

Even just starting with 'identity' for a web app, i.e. Facebook and Twitter integration along with basic credentials - can be a big pain.

Partially I think this is a question of what you mean when you say "app". There are millions of websites running Wordpress, plus maybe a specific plugin to do e-commerce or something. They are very close to the ideal you are describing.

There are also millions running on things like Shopify, Wix, etc.

Usually these kinds of things are not really made by developers, or at least only require a very small amount of development time. It's only when we get to the more complex 20% of applications that we start having a "problem".

So I think when you talk about most apps being basic, you're right. Excel allows people to do in 10 minutes something that would've required dedicated applications built by trained professionals 30 years ago. We're just not stopping there.

"Out of the tarpit" (http://lambda-the-ultimate.org/node/1446) is an interesting contrast / comparison to "No Silver Bullet".

It's not too easy. I've worked on one such app, React Studio:

https://reactstudio.com

It's maybe 80-90% of the way there, depending on what you need. Closing the gap feels demoralizingly hard with no resources (I'm spending time on it unpaid).

The design environment is quite complete, although it doesn't even try to expose all of CSS -- instead it's more like a "walled garden" of the parts that make sense for mobile-first designs.

I'm most proud of the exported code's quality (no extra junk, no giant library hanging around) and also the component and plugin architecture. You can either design React components visually or import them from code (e.g. an npm package), and both will work the same within a design.

The fundamental problem is that the technological foundation for web apps is so unsuitable, like building a house on quicksand using decayed planks. CSS is a cruel joke with its thousands of properties, their decades of cruft and their unpredictable interactions. HTML was designed for documents, then was semi-accidentally turned into a view layer, but it never adopted any of the learnings from the successful '90s GUI desktop frameworks. Some of that missing structure has been added back by efforts like Angular and React (and the latter is fine to my taste), but it's just really frustrating that web front-end development is still stuck endlessly patching over a rat's nest of <div>s into something that resembles actual UIs. But what can you do? It's not like browsers are going to add a real app framework. So here we are, still trying to decide whether buttons should be <a> or maybe <input> elements. Sigh.

There are two reasons we're here:

1) For the end user, it's too hard to download and run an app. 2) For the developer, it's hard to create an acceptable multiplatform app.

And platforms can be moving targets. New platforms are introduced and adopted, along with new ways to interact with them.

The platforms can also change and break applications in unpredictable ways. For example, iOS 11 won't run 32-bit applications.

I think what you mean is that Apple platforms will change and break applications in unpredictable ways.

> So here we are, still trying to decide whether buttons should be <a> or maybe <input> elements.

<button>, obviously =)

Keep up on the good work

The issue is that each piece of customized functionality is different. It also changes through time. Much of the functionality is standardized. Many of the places where customization is expected, hooks are built into frameworks to allow easy extending.

Designers come up with new rules that do not fit into the mold that the "automated" tool offers. For example, tool tips that appear only if the ? key is pressed on the keyboard before hovering. Custom code is needed.

The customer decides that they don't like the ordering, content, or display of the timezone field. Custom code is needed.

The law changes and it in addition to "Male" and "Female", additional values are required for the gender field. Custom code is needed.

Country XYZ decides that your software cannot be used as is because the country dropdown contains Fakeystan. Country XYZ considers it part of its territoy. Custom code is needed.

There are analogous situations in the real world. However, they are encountered less often as we got used to accepting the standard offerings that are available. True customization is rare. When we build a custom house, most of us decide between one of the handful of standard options that are available on the market. Picking and installing a standard toilet with one of the myriad toilet seats available in the store is easy. But wait until your you get asked to install a plus-sized purple Italian marble bowl with a faux-fur seat that is anti-bacterial, easy to clean, and doubles as a floatation device. These are the problems we are dealing with building software every day.

Because then you could only build very simple applications where your data model was extremely tightly bound to your application and user flows.

Once you need to do things that require pulling from many DB tables and rows in your views; enforce complicated logic about who can see what; and enforce complicated logic about who can do what, the automation will get in the way.

The thing with manually writing SQL queries and data access code is that it doesn't take as much time as you think it takes. The problem with tools that automate this is that they can be buggy, or have a steep learning curve in order to generate a reliable and high-performance program. SQL queries and data access code is very straightforward and easy for the next guy on the project to maintain.

With the hidden cost of learning, debugging, or working around flaws of automated CRUD systems, it's easier to just take an extra few hours, or few days, to "do it right," at the beginning of a project. Otherwise, you risk spending many days trying to work around problems in your framework.

But, think about it this way:

- Ruby on Rails is popular because can build an entire CRUD web application around a basic schema / data model. You start with your CRUD fully running and enhance from there. Most of your time is spent enforcing your business rules and making it look pretty; (or working around the fundamental performance issues that come with such an environment.)

- Excel is popular when the user doesn't need the UI. Very complicated business processes live in Excel spreadsheets. So, if your use case is so simple that an application could be auto-generated, Excel might be "good enough" instead.

What if it is 90% automated? As other pointed out, see Django REST Framework, Rails, Elixir, Scala Play, and enterprisey frameworks like SAP, Odoo, JDEdwards, Oracle Whatever™, ENOVIA, Windchill, etc.

Those tools/frameworks cater for the accidental complexity of "writing CRUD stuff". Remains the essential complexity of what makes your app what it is. The business logic for YourBusiness is not Uber's, is not Greenpeace's, is not Pfizer's, is not Ford's. The logic may be mundane, but that doesn't change there's nothing "natural" to it, so it has to be expressed, and that's what we do using code and the tools at hand.

As for the percentages: sometimes it's gory and you do everything by hand (where you manually and painstakingly write yourself 50% of this non-essential, non-value-added 90% accidental complexity due to lack of tooling), then by all means go grab a few shiny shiny tools for the next project, and some other times you do have tooling to cover these 90%; remain questions to be asked to your users, specs to derive, and code to write.

I'm guessing you mean Phoenix where you wrote Elixir. But yeah, I immediately thought of these frameworks and how fast you can get from zero to CRUD API with them. Phoenix definitely felt 90% automated, Django and DRF had some extra glue-work last time I set them up. Not sure about now.

Anyone tried API Star by Tom Christie? Seems pretty interesting and relevant: https://github.com/encode/apistar

It used to be, back before SPAs were common, about 7-10 years ago.

Rails, django, asp.net mvc - all of them had autogenerated most code, including html, handled auth, persistance and such.

Even now if you don't go spa it's very common.

Spa changed things considerably and the stacks haven't caught up in maturity yet. But we are on the way there, especially node.js and react based stacks.

Firebase is doing most of this for SPAs: auth is standard, data persistence is done with declarative rules (basically no written backend), and using Create React App for views handles most other things.

I’m actually working on a boilerplate now to show how simple dev can be with these tools:

http://getfirefly.org/

I love the idea behind firebase, but it's just not worth it unless they have triggers - even the simplest functionality is extremely hard without pre-persist triggers.

Could you tell me a little more about what that means? Would like to hear what I’m missing out on.

Possibly relevant: https://firebase.google.com/docs/functions/firestore-events

These are async functions - which means they happen after the data is already in the database. They basically act like a producer/consumer pipeline.

Triggers happen before the data is saved - which lets you change the request (or other data in the db) before it is saved or abort the request.

Without it even trivial operations become hard - such as add a column from other places in the database, incermenting counters/cached aggregates, authorization that isn't role/access based, etc...

Ah, gotcha. I think I’ve been getting by with Firebase transactions for counters and db rules for complex authorization. Firebase doesn’t have columns.

Can you share a specific use case that’s difficult to accomplish in FB?

I have been working towards this goal since the last 5-6 months. Some documentation is up here at:

http://docs.dapt.in

Source: https://github.com/artpar/daptin/

The CRUD api is based on JSONAPI.org standard to make use of their available clients (http://jsonapi.org/implementations/)

For the UI part, there are multiple things in place

1. First, upload via ZIP to one of the storage backends ( rclone is plugged in so storage backend could be anything from gdrive to local file system)

2. Grapes JS for UI drag and drop designer after you have uploaded (http://grapesjs.com)

3. A detailed authorisation framework to control entity level and row level access

4. Actions and Outcome chains with complete JS support to handle business logic (for example the signup/signin flow is a set of actions)

ps: earlier i used to call it "goms". recently renamed it to daptin. Also, not focusing on other parts of daptin to keep this short.

> Why Isn't the Construction of CRUD Web UI's 90% Automated?

It is, several times over.

Using SQL or some other domain-specific query language against structured records? That's the first 90%. HTTP and hypertext? That's the next 90%. CSS and modern HTML authoring? Another 90%. Web app framework with an ORM and MVC / data binding / reactive / flavor of the month? Another 90%.

Your mistake is assuming that automating 90% of it reduces the work by 90%, rather than letting you spend 10x as much time on making the bespoke parts fancier.

Not quite sure I follow. I'm just thinking about how to deliver value to users faster.

I think you're asking the right questions. And it's possible that you're onto something. But consider this:

Often the way the data is represented in the DB is very different from how a user thinks about the data in a given context. And another user, in a different context, might again think very differently about it.

So you need a lot more configuration for the input data than you might think at a first glance.

Lastly, there are good frameworks already that do a big part of the work.

Django for Python is one that I'm very familiar with. If you combine it with vue.js and bootstrap on the front end, it's very quick to create a working MVP.

It seems that folks at HN look down on so called "CRUD" apps. But the truth for most CRUD apps is, that a lot of thought went into building these apps and they have a lot of important parts, whose construction cannot easily be automized. Good examples are:

- Certain inputs might trigger other actions (integrity checks, sending an email, notifying another users, etc.)

- The UI / UX needs to be optimized for the specific Usecase of the app

- There are unlimited options for filtering or ordering items, and the choice of options depends on the specific Usecase

- A lot of apps need Cronjobs in the background to support certain workflows (notifying a user if a certain input has not been done in a certain time)

- There can be rather complex rules on who is allowed to create, update, delete or view certain items

- AB tests and Analytics need to be supported for a lot of Tasks / Views

- etc...

I cannot imagine a "CRUD" framework, that supports all of these issues. It would basically just be another programming language ;)

1000% agreed.

I've actually written something (not open source) that largely solves this problem for myself and my company, but, its a very hard problem to solve in general. The largest benefit that I find from it is that it allows you to make some things that would done directly (EG: UPDATE statements) be by a web UI. Figure if you had a particular table that only got updated once or twice a year, is it worth it to write a bunch of CRUD code for that? No. Is it worth it to write 20-ish lines of JSON to describe the table and avoid a bunch of problems that direct DB updates cause (accidentally hit enter before I typed where on that update, no tracking on who makes what change when)? Yes.

Has a lot of benefits. I understand its a hard problem, I don't understand why no one (aside from perhaps django-admin which I find too obtuse) has tackled.

I may rewrite existing code to golang (PHP now) and open source it at some point in the future.

I love the premise of Wavemaker: Write your CRUD app visually, and add your sprinkles of business logic with Java or Javascript. It's a shame it's closed source and incredibly expensive ($250/mo in the cheapest tier).

I never understand when people wish to build a site they must implement the login page. There really is no page like the login page. It should be as standardized as it gets. If you were doing a site in rails some poor nob is going to sit there and code the password and login logic - sure there are gems but they aren't maintained or don't work with your hipster react plugins. You could whip out drupal/Joomla!/wordpress have everything done for you - and worry about skinning and which plugins to add. But that would be in php - and using php is the technical equivalent of being called a child molester. So where ever you go, whatever new company you work for - some tool wrote their own login page. lion king Hakuna Matata.

Business logic. +if you only look at the data side, most business objects don't have the same shape as database rows, yet most CRUD programming is tightly bound to table representation.

You could do the transformation before it hits the ui, and for ui specific business logic, call into it from the higher order components that wrap the common components provided by the community.

This is the job of a model. Say I want to manage videos and their rights as they relate to territory, time, and user; or I want to create contracts between my customers and financial exchanges allowing them to access specific data feeds, each with its own additional contract, regional and volume usage limits, and SLA's. And these are synthetic simplified examples I just pulled out of the air - real life is way more intricate. Let's not even get into multilingual and versioning for audit trails.

What I don't want to care about is storage implementation to implement those models in a given channel/client. If that starts leaking through the model abstraction layer into implementation, my business rules aren't testable, I can't change backends to scale without rewriting all my implementations, all of which are going to be bug-ridden and spaghettified.

The fact that we would even describe storage as a transformation (as if it was just a mapping with some consistency rules), should flag up how brittle such an approach is going to be. No wonder it is so painful for businesses to make any changes to their systems.

Now having said that there are many simple cases that do map well to a table. Those are already managed directly by businesses without the assistance of developers - in spreadsheets.

I had a client ask me this not so long ago.

So I went through their requirements and separated the "mere CRUD" from all the frilly extras, novel integrations and look and feel requirements, as well as translating their actual business requirements into the "CRUD space".

The mere CRUD bit was very much a minority of the work that was required.

Great question.

So I'm kind of obsessed with this problem, believing that programmers in our society could be dedicated to much more than building the same application over and over again.

If your page is a listing of items (could be e-commerce, could be a listing of users in an internal application), it's really just a fancy way of saying List<T>.

At the same time I know that some things can't be automated well and still require translation from spec to layout. Some flexibility is required.

However, there can be a starting point which can be built up from. Then components can be defined for things that need more definition, that need a well-designed UX or specifically tailored user interaction design.

Where I am now is a small template language that also contains the definition of a reducer-based dispatch mechanism.

I am close to having TodoMVC implemented in about 80 lines of this scripting language, which can be compiled to javascript using any number of frameworks or libraries, but can also be used to serve a functional site with the HTML pre-generated which can then be hydrated with client-side javascript. This is all done without using server-side javascript.

I have designed it to be usable by someone without an ML background, someone who doesn't fully grok monads or lift.

It is mostly imperative with a functional pipeline/filter construction in places.

You can see an older version of the code and a static demo here:

https://tmzt.github.io/isymtope/

You can also see where it's going:

https://github.com/tmzt/isymtope/blob/feature-querylens/res/...

https://raw.githubusercontent.com/tmzt/isymtope/feature-quer...

I think Django's admin interface fits what you want (except maybe the invision-like tool for design prototype)

I have some experience with the Django admin and I think it is actually a good illustration why user interfaces are still complex:

Because the way we abstract our data in the database is often very different to how we think about the date in a specific context.

I think this is a good example of why we aren't close to a web ui of sorts to do away with 90% of coding. As good as Django is. Usually there's still a lot of coding required.

It is indeed silly to write the same type of code over and over, and that's why people automated it 10 years ago. This is exactly why Rails made such an impact; it automated the boring stuff, and I think Rails does pretty much everything you mention. And what Rails did for Ruby, Django then did for Python, Grails for Groovy, etc. I suspect there's a framework like this for every language out there.

Of course some websites need something different than CRUD, and especially with the modern trend of SPAs talking to JSON APIs, not everybody needs a simple CRUD website, which is why there are also lots of other frameworks. But if you quickly need a simple CRUD site, the tools still exist.

If those tools are not good enough, then build on them and make something better. There's always room for improvement. But the thing you're asking about has existed for 10 years now.

I take it you've never used Rails' scaffold generator ... it does much of what you speak of (db schema, basic html/css forms, etc.)

Sure, but going from there to doing simple stuff (e.g., handling a file upload, authorization, client-side validation, client-side form modification [e.g., make a selection on this dropdown and this other dropdown changes], making the app multi-tenant) requires a lot of additional code, which should, IMHO, be part of the framework.

Hmm, ask Equifax how simple file upload is.

It's actually a pretty complex problem to solve securely, leading to scripts that try to support every configuration including off-the-shelf PHP hosting.

An ideal would be something like S3 where invariants like size, type, file format could be verified at time of upload and then exposed in on-demand manner to the rest of the application.

Exactly my thought. It shows that Rails really grew organically from real-life business patterns as a way to automate the mundane. The Rails resource generator generates views, controllers, models, migrations and routes.

That sounds useful. The thing is rails is just one of many frameworks. What if you're using a go backend for instance?

Also, does the generation output single page apps for popular frameworks such as react and angular?

Then you'd use another templating language for those languages or a generic one like m4.

This is what we have so far: https://varick.io/

We make a ton of MVP apps, and what you're talking about would be amazing! But the reality is we'd never get close to it. Every app focusses on a different audience. We aim to help them reach their unique goal as quickly as possible, which inevitably means a different UI approach for each scenario. It's not worth the effort in trying to automate.

Instead, we focussed on a generating a front-end for admins. It works with APIs so any business logic and security validation is just baked in cannot be by-passed.

There's a new big trend in tools for enterprize(and a bit of consumer) web dev, called low-code/no-code. those tools offer that level of automation, plus much lower barriers to implementation(mostly via visual interfaces), and it seems (but i may be wrong), that they can be used to built complex systems(often by software guys, complexity reduction and order is still important).

But those are almost never mentioned in hacker news, which may say something about our community more than it says something about the possibility of building such tools.

Oh cool, do you know the names of them?

https://www.pcmag.com/roundup/353252/the-best-low-code-devel...

I think I worked with one of those, the pre-C# version of Metastorm BPM (business process management). It was like JIRAs workflow feature with more flexibility - permissions on who edits which fields when, and transitions could be triggered by a button or external event. It modelled a lot of business processes well and let you pre define searches and views for specific job roles. You laid out forms with WYSIWYG.

The main downside was the programming language - for an if/else statement, condition and both blocks all had to be on a single line. Variables and functions could be referenced in double-quoted strings meaning the syntax highlighting was always wrong. And there was no chance of loops or user defined functions. This meant code editing was done with repetitive clicking in the IDE.

The other downside was inability to diff or branch. It kept a history of deployed versions and that was it.

The same applies to mobile. 80% or so of mobile apps are just listviews with a JSON backend. Why do I have to write a list adapter, syncronization logic, lifecycle management etc. again for each app?

Ideally, you should be able to say "Here is my schema, gimme a listview, keep it up to date. Here is a template for the list item, here one for the details view." Make everything declarative. One crucial point is to then make it easy to tweak everything - often abstractions make simple things easy, but slightly less simple things very hard. Once you want to do something more complicated, like having an animation transition from one view to another, you shouldn't have to abandon the framework, but should be able to drop to native code.

And yes, on the web we have Django, Rails, etc.. But I still have a lot of repetitive work to do. Why isn't there something like the Django admin which is more suited to users than administrators (I'm sure there is, but it is not mainstream). Let me define my schema, press a button, and I have an ugly CRUD app. Then add constraints and business rules, and pretty templates one by one. Something like, I don't know, SAP or MS Access - a glorified DB browser which you can customize a lot. Add in a graphical designer so you can let an intern create new forms and tables.

I seem to never find areas with which this quote doesn't apply: "We shape our tools, thereafter they shape us."

In short I think web developers build with the tools they know are capable of delivering a spec but this does not necessarily mean those tools are the only capable tools for some inherently unchangeable reason.

If some tool comes along that can allow devs to scaffold out 90% of a complex web app without sacrificing fine-grained control for the remaining 10% it could be a winner. In my mind, however, this tool would also have to weave itself into scalable practices like git workflow, modularity so many devs can simultaneously work on one app, etc.

I think the reason there have been so many attempts at such a tool without great success is that engineers building such a tool quickly bump up against a multi-layered, exponentially complex universe of edge-cases and to tame that complexity alone likely requires an enormous amount of careful, considerate and cohesive decision-making. And once you throw in the fact that any startup will be asked to deliver an MVP ASAP you start to see why the odds are stacked against such a disciplined approach.

Where I work the codebase is massive, 100's and 100's of internal modules all working together in many "sub-apps" with teams all across the world. One task was to add a field to a form and it took 5 engineers across 4 teams 2 weeks to even get it in a testing stage. Maybe an app this size is not the best target for such a tool but if it were it'd have to account the massive operational complexity as well, the human processes at scale.

This has been a thing with ASP.NET/Web Forms within Visual Studio for a while (C# or VB.NET)

Since the views can be composed with a master view, the actual code generated is just a table element with a loop to add the rows/cells.

Although the view model is usually a plain C# class object, those objects can be generated from looking at the DB schema, meaning you only have a two-step process.

You could generalize it to make a table for any C# object using reflection, but that's kind of abusive IMO.

I haven't used web forms or asp.net mvc in a while. Do they output react or angular now? Do they not require a .net backend? Those are hard requirements for a lot of teams.

They can certainly work with Angular/React. There's templates to get started with that. It would be a .NET backend, but a .NET backend isn't exclusive to Windows any more with .NET Core and related Docker images.

2 years ago, I had exactly the same question, especially for the internal tools (e.g. the admin interface). Some frameworks (like Loopback in Node.js) have the ability to generate the CRUD API automatically based on your data schema. The downside is the way to customize it. IMHO, it's not "easy"/"developer-friendly" enough to convince me. Back to the admin interface topic, there're also many tools (ActiveAdmin, Django Admin, Sonata, etc.) that automatically generate the CRUD interface. It reduces a lot the maintenance, but one more time, customization is limited, DSLs are just ugly and they are not SPA :( I never understood why it's not simply based on an API.

This is why I've started Forest Admin (https://www.forestadmin.com). It's an open-source library you install into your app. It automatically generates the REST API based on your data models and it covers 90% of admin operations (CRUD, analytics, etc.) and let developers customize the logic (custom actions, computed fields, complex workflows, etc.) without learning a complex DSL. In addition, we also provide to business team a customizable UI as a SaaS.

Because developers & designers prefer to have control over their HTML, and HTML really isn't that hard.

Any attempt to create a true multi-purpose UI builder for web applications has resulted in something more complex and hard to use than the problem it was trying to solve.

Case study:

Rewind 10-15 years and Microsoft's ASP.NET WebForms was popular (subjective use of "popular", I know), and was designed to give pretty much what you are asking for. It really did work and was great in many ways.

However, it achieved this was at the cost of a _lot_ of proprietary complexity (search for "ViewState" or "ASP.NET page lifecycle"), which over time became burdensome. Then AJAX came along, and WebForms was updated to achieve this but with even more complexity (search "ASP.NET leaky abstraction").

I was using ASP.NET WebForms extensively around this time and we found ourselves increasingly "fighting" the framework, trying to bend it to suit our specific requirements (which weren't all that radical). Also, our design team started delivering their work in the form of HTML and CSS, which was a nightmare to integrated with WebForms as it dictates the exact HTML that is used at the UI control level.

Then we moved over to an MVC framework, and were enlightened...

Right, I used webforms back in 2002-2005 as well. Not the best experience.

I am talking about modern single page applications that leverage xmlhttprequest and common frameworks such as react and angular.

Removing the viewstate and using more ajax are merely a modernisation of webforms. Fundamentally you are calling for what webforms was actually doing, ie generating a UI, binding it to a database and dealing with the boiler plate code in between.

The problem with these all-in frameworks is that they are very good at giving you a fully functional and OK looking app in a few clicks. But from there they work against you. You don't want to expose everything in the database to the user, or you want to include some custom validation logic, or you are dealing with some data that is partially normalised, or you are mixing SQL and noSQL data, etc. And then you are fighting the framework to hack something together that the designers didn't intend. And after a few years you realise it is less work to do it yourself with a bunch of helper methods.

I think there are two answers here that also point out a couple things about the question & its framing:

1) Because almost no project is just CRUD Web UIs.

There are always so many small and big details that deviate from the CRUD Model. Unless you're building a Database UI. Which almost nobody wants as people want software that solves their real world problems - which rarely map 1:1 to a DB and it's schema.

IMHO frameworks that optimise the CRUD sacrifice a lot with regards to simplicity.

2) Plain CRUD Web UI is 90% automated, BUT: see item 1.

Rails and Django will handle almost everything hard for you and you just need to declare some trivial stuff like the model, url and presentation. The reason why you end up coding so much even in a Rails/Django project is because item 1.

Also, Rails and Django are definitely frameworks that sacrifice a lot in order to show off an easy basic CRUD setup. Both have ton of horrible OOP and metaprogramming under the hood and you'll have a tough time understanding what they do and how.

That being said, I've worked with both and I do like them, especially Django.

Simply put, because it's much quicker to write it from scratch each time. A good developer will spend time to learn all the little things like how to make a table, how to store data savely, etc. And he will add these features as needed, and in a form that matches well with what the other pieces will be. It is really normal once you've hit that level of skill in one area in your life. It can be something totally else entirely.

One could also ask why a painter doesn't use puzzle pieces instead of drawing each picture from scratch. Sure, it would make him finish much quicker, but it doesn't give him the level of detail he needs. And if you make these puzzle pieces small and flexible enough you might call them pixels and then what's the difference between using them and just drawing by hand?

What you will see often though is that people use a framework to get some level of detail out of the way. But often the good developers will also be able to take out whole chunks of this framework and replace it with own code.

Right, abstractions are useful. I'm just curious why we keep recreating the same abstractions over and over again.

You need to interact with the question, why a painter won't use puzzle pieces and instead draws fresh strokes each time he draws a picture. That's the core point of the idea and I think with the example painter it's much easier to see than with programming.

And I also answered your question already in a literal sense. We don't really recreate the same abstractions. They feel similar to the user, but are quite different through having a detail different here and a detail different there. E.g. in one CRUD app you load the data from the web API, then filter, parse and format it for presentation. In another CRUD app you already filter and transform via SQL in the database. This is not done randomly but for a reason. Through the whole process something different is created each time. At a certain skill level you simply want to have that flexibility.

    Behavior should be mostly auto generated config

Key word there: MOSTLY. The trick that's never been really figured out is how to make adding the bits that can't be generated on top without making a brittle mess.

If you squint a little, you could also regard the Wordpress ecosystem as getting close - I've seen things done with custom fields that are remarkable.

Right, to me higher order components could get us there.

Because you would get 90% of the way there, and then have to start over without the automation if you wanted to close out that last 10%.

That's a good argument. If it used hand coded html and CSS, and had a huge community of fully customizable components where you had higher order components to enable the customization, what would the challenge be for the last 10%?

My guess is because UI is a massive source of churn. Anything which is visible to a user and/or customer will be under a constant barrage of changes from customers, marketing, management, etc.

One reason for this is, of course, that it's actually visible to them (unlike, say, a communication layer between subsystems, which may sit there unchanged for decades).

However, there's also the common perception that all software gets built from scratch, as if it were some bespoke piece of furniture. Hence anything generated, or using "off the shelf" code, will still get bogged down in change requests, not necessarily because those changes are "important" enough to justify the effort, but because "since we're building this whole thing anyway, we might as well do this part this way" (not realising that we didn't build this whole thing, and it's the changes which are the actual cost)

I think things just move too fast in the web world in the past few years... Yes, a common component plugin standard so you can reuse it in both React and Angular would be nice (Web Components?) but remember not that long ago, we were debating between EmberJS, BackboneJs and jQuery UI. And Gulp vs Grunt, and UMD vs CommonJS, Bower vs NPM. React and Angular4 are very, very new relatively, and they evolve pretty fast, with breaking changes. A company betting on building a CRUD / UI builder needs to go into market in a few months or it's already behind.

Take a look at react studio (see comment here). They are a great tool probably, but they are focused on one popular library at the moment. The change is so rapid that by the time you create an automation tool, it's generating code that will be "obsolete" in terms of people who would like to maintain it (would you go work for a startup where you need to maintain backboneJS or jQuery UI all day?). The alternative is to generate code that doesn't try to be "latest and greatest" at all, and not really care (see wix code post earlier today)... but not sure it's a great bet as well.

I think back in the day, there were just less options to choose from when building apps, and they evolved slower. This is why in the desktop days we had a lot more robust UI builder, app builders (Every IDE has top notch visual destkop UI editor, but also good old Magic, MS Access, FileMaker etc...).

I do think this is begging to be changed. And I'm sure a few people have this in mind as well. But it will always be one step behind of the latest web frameworks.

However you do have some solutions

1. Java - JHipster uses yeoman to generate Spring Boot + Angular apps including basic CRUD UI

2. Rails - rails scaffolding do basic CRUD nicely

3. Node.JS - http://loopback.io

I'm sure there are more. And I'm sure there will be more. But in the end all the above do is give you a jumpstart, if all your app is doing is just CRUD, then your barrier to competition is already low, whether it's automated, outsourced or done by Stanford grads getting 200K a year.

But I do think that eventually your idea of shared components should be a thing. Just need to get google and facebook in the same room and agree... weirder things happened... companies agreed on standards before...

That's an extremely good point. The solution to this would be that the output is plugin based. It could spit out react, angular, next framework, framework after that etc.

Edit: Thinking some more about this. I think the future proofing is the hardest aspect. If you were able to leverage a custom compiler to transform react or angular components to a future framework, that would get you there.

Also, Jhipster looks really great _if_ you happen to be using java.

Jhipster looks really great. Are people primarily using it to just bootstrap a project or actually using for the duration of a projects full lifecycle?

> There have of course been hundreds of UI builders. Why aren't they used?

I think the reason is that building something with these builders may be easy at first, but when you want to do something a little different than foreseen, the cost goes up enormously.

It's this "cost gap" that developers are so afraid of, also because it is very unpredictable.

With today's web frameworks, you can set up a simple app in an hour. But that simple app probably won't solve anybody's real problem. There's always some complication.

Right, a solution would have to allow customization via a 100% code approach to be useful.

The root of the problem is that we tend to need special case automations to build UI for the same reasons that carpenters and blacksmiths need jigs and molds, but we keep reaching for generic tools to solve them, because software lets you extend things indefinitely and so an "ultimate" generic tool is always just out of reach.

For example, instead of a one-step translation of "invision mockup" to "final site" it would make sense to translate the mockup into an automation conveying that specific design language.

HTML and CSS deceive us by apparently being those generic tools to solve everything. But because they're generic and composable in a high level sense they actually get in the way of the intention - layout, content, and interactions are all shoved into non-overlapping functionality boxes. And then we add JS on top of that, making it possible to break out of the model badly.

I see what you mean. Doesn't component level style and markup encapsulation solve these though?

I've been prototyping a tool to generate most of the implementation whilst still allowing for BI and design changes. Would love feedback on if the premises make sense.

https://github.com/hofstadter-io/geb

Can you explain what the "hofstadter framework" is?

Found this from the author's GitHub profile: https://hofstadter.io/

You're asking several different questions under a general umbrella which have a lot of specific and long-winded answers, but to be succinct about it: business want incredible amount of customization in all different ways, and customization is resistant to automation and patterns. Successful frameworks straddle a good balance, which is why they have staying power.

I think if you push your question to the extreme, we can ask why Literate Programming never took hold. Wouldn't it be vastly more cost effective for businesses to completely commoditize the entire IT department with just a specific kind of technical writer who could pass along these specs to a service that would compile the application as written? Competitive technology simply isn't that predictable and circumscribed in capability.

It used to be.

The grails web framework was capable of generating relatively complex web ui from the defined domain classes.

It was super cool! Sadly, it's not so used anymore, it seems quite dead. Last time I tried to use it I found out they switched to gradle, making its usage twice more complicated than it used to be.

>Behavior customizability should come from tweaking the auto generated json config, and adding some HTML and JavaScript to higher order components.

Show me a single component on any platform which can be customized via properties to "do everything".

That property does exist and it's called Turing completeness.

There have been countless attempts (http://thedailywtf.com/articles/The-Mythical-Business-Layer , http://thedailywtf.com/articles/the-inner-json-effect , http://thedailywtf.com/articles/The_Enterprise_Rules_Engine to name but a few) at trying to reimplement a fully-fledged programming language in XML / XSLT, JSON or whatnot. Ultimately, all of these attempts seem to end up being more complex and unwieldy than the tools they were intended to replace.

The OP calls for a tool that allows you to automate most of the cumbersome work involved in building web UIs. Perhaps programming languages just are the closest abstraction that still allows for a sufficiently large degree of freedom when it comes to accommodating specific requirements.

After all modern programming languages and frameworks already automate most of that really arduous work. They allow us to define data-driven applications in concise terms that for the most part already resemble a specification written in natural language.

More accessible languages that more readily provide clear feedback (think of Bret Victor's work for example) could be a way forward.

To do everything, you write your own extension of the base component as a higher order component with custom code.

Not entirely sure what do you mean by “higher order components” but what you seem to describe would lead to the same problems we had with inheritance in various oop languages.

https://reactjs.org/docs/higher-order-components.html

I dealt with this quite a bit and recently consulted for a friend wanting to do something similar.

The issue is you think you can drag and drop data fields and be all set. But to deliver a halfway decent UI, you need custom bits and pieces here and there. From layout, to extra JS, to new "picker" types for some fields, and on and on.

When I've looked at it, the issue is either the forms toolkit gets stupidly complex or doesn't have a good customizability story, and in frustration you throw away the toolkit and just do it by hand.

There's room to do something better, so long the customization is really easy and trivial to add one-offs.

> There's room to do something better, so long the customization is really easy and trivial to add one-offs.

Exactly!

What do you need beyond something like React Forms with additional HOCs? I'm curious what you found.

Because businesses are greedy and short-sighted. They all think the problems they're solving are novel, and they keep deviating from standards and reinvent the wheel as a way to diffentiate themselves from others.

Truth is, most businesses are solving the same problems and a single general purpose platform could easily be used by all of them. I'm not talking about a framework that all businesses could use to build their apps, I'm talking about one app that could be used to interact with all businesses.

I am convinced that a team of 10 people could make 80% of existing software (and tech companies) obsolete in about 2 years.

I've asked myself that many times before and I'd say it's because it's not an easy task and often you need to get things done to meet deadlines, instead of developing a general solution, that although could save time in the future, might not help much in the short run.

Having said that, Django admin is the best example of that. I'm a huge fan of it and tend to use it on every project that fits a crud workflow, mainly business apps.

To give an idea of how much work goes into building something like that, it took 5 devs an year to build its first version, not to mention add new features and improve it.

Django admin sounds like a very useful tool.

What is the closest thing you can get on front-end webdev to something like Windows Forms, WPF or QT Creator?

Ie something where adding a button or a dropbox in a specific location does not require intimate knowledge of CSS tricks while at the same time producing reasonable code.

I found old Actionscript more consistent than modern webdev.

What I mean I do not need to know advanced XAML to produce quick and decent CRUDs on Windows Presentation Foundation, while for the web this means hours spent reading up on various quirks.

It almost means that you need a dedicated front-end guy with you all the time if you want to iterate your design.

I also ask the same, why webdev doesn't have tools like that, why everyone love to write a ton of css to make a button on the middle page, See struct, it's a wysiwyg gui builder for react

There are a handful of Rapid Development frameworks which will automate the majority of CRUD. My personal favorite is Web2Py. I can build a working protoype of most ideas in a week or two with it :)

Is that faster than just doing it with Django?

I would say so. The biggest difference I find is that Django makes a few too many assumptions and can be a pain to tweak the way you need.

Web2py was created because the author found Django too difficult to teach to his students at DePaul University. He rebuilt Django while fixing many issues in how it was designed, and simplified the API significantly in the process.

Instead of Django's admin interface, web2py gives you a Grid component which allows you to create an Admin interface for any model/relationships that you like within any page you want. So your Admin pages are just another route/controller which use the grids as necessary.

Web2py's ORM is super nice to use. It's very similar to using plain SQL, and incredibly powerful. It holds your hands a bit less than an ActiveRecord style ORM but I find it also traps you in fewer corners. Especially when it comes to reports/aggregating data.

Likewise, it's a pain to set up email addresses for the user's username in Django, or to add custom fields to the User model. In web2py, it's dead simple.

Web2py has an easier to use authorization layer than Django's, and it handles migrations more smoothly.

Web2py's HTML templating language is much nicer than Django's. It's just python, nothing extra. Super easy to pick up.

Web2py forms support file upload fields, and image upload fields out of the box. It also has a module for automatically resizing image uploads as necessary. It includes a datepicker for date fields, and it also has the "LOAD" function which makes is super easy to Ajax up your site.

The other big thing that I love about web2py is that all the form/grid helpers output Bootstrap compatible HTML out of the box, so you can throw on a theme from Wrapbootstrap.com and just like that you have a beautiful site.

And Web2py works well with Intercooler JS so you can get quite far without having to write any JS code.

It even has its own alternative to Intercooler built into the framework: http://web2py.com/books/default/chapter/29/12/components-and...

It's super cool :)

For it to really work you need your environment, tool chain and all components to be fully integrated, which significantly limits ones freedom when choosing the frameworks and libs. Last few years front-end is evolving at a such crazy speed that it seems no scaffolding tool was capable of keeping up the pace. You probably can find something for technology X, but as soon as you try to combine X with Y and Z, you're much better doing it by hand, or by combining manually the result of a few simpler tools.

As one of the dozens and dozens of people that not only do CRUD for life but kind of like it:

Is almost 94% automated.

The name of the tools: Relational Database System.

You need to "just" describe the data, relations, queries and perform the input of the rows. It automagically give the results you want, and kind of fast most of time!

---

The "Why?" CAN"T BE fully automated (or close to 100%):

How many program you can fit in this description?:

Create Read Update Delete

Like, all of them? (given or taken a few dozen???)

---

The "HOW" we could actually do CRUD more nicely?

You must put aside the OOP model. It could be there, but aside.

The functional neither is. Nor lisp, haskell, or anyyhing similar.

You need a truly relational language (remember how a RDBMS is the answer to your question?).

Probably need to plug a actually nice Tree-manipulator language (exist a relational model, but for trees???). OO is almost there, but not close enough.

You can mix OO and anything else.

This is not a pipe dream or untested. The dbase family operate like this. I code for work in FoxPro and was FAR better than Python, Delphi, .NET(F#) (all I like, but HELL NOT, THEM ARE NOT GOOD DATA LANGUAGES)

The other side of the equation is the form/report builder.

Let me tell you: You ACTUALLY NOT WANT A (auto) FORM BUILDER.

You need a FORM BUILDER DESCRIPTOR and the flexibility of build your UI manually based on this.

THis is not speculation or untested. This was (almost) how FoxPro operate... the descriptor was tables. The tables were used to build the UI. Literally the UI was serialized on tables. Like how cool kids this days use JSON. But potentially better if you discover this (ie: Open the form file and see is just a table!) and then use this for the full effect.

Obviously, FoxPro and other tools were rough like you can expect for "ancient" tech. But similar how Lisp, SmallTalk, APL, Rebol, Pascal, Modula, ML were "rough" and we are rediscovering the potential them have, this is the same with the dBase family.

I've felt like I'm so close to a powerful library that will be robust and advance my pursuits quickly. Hasn't worked out yet. I am getting better and better. Frameworks and libraries are getting better too, but to remove something like 90% of a CRUD app that isn't literally just a basic content CMS site - I don't see that happening soon.

There's a movement called MDA (model driven architecture) which aims at large scale automation. I regularly keep an eye on this, but it never seems to leave academia. There's a lot of leverage that's possible but it's hard work to get there.

It is mostly automated to an extent. A previous company i worked at, one of our main focuses was making a component out of everything so when it came time to build a new page, we just pass a config + dataset to our component and that's it voila page built.

This is one of the reasons I use Drupal. It is bloated a bit but the CRUD system is out of the box easy. You can add properties to entites on the fly without code, then turn it into an API system in a matter of minutes.

Well you can do it badly and end up with ... wait for it ... ScriptCase. You know you crossed the edge to insanity when the Login form consists of 10,000 lines of generated, ugly looking code.

See Ruby On Rails.

http://rubyonrails.org

You will start fighting Rails as soon as you add a frontend framework. Rails was conceived in the era of server-generated webpages.

It will work just fine with angular or react. I have seen nothing comparable in the era of JavaScript frameworks.

Me neither; I use Rails all the time. But that doesn't mean Rails and React are made for each other.

It's so funny you readily presume JavaScript as the ultimate web language. It is everything but. The only reason people still use it because the browser holds them hostage and then they readily march into the hell that is server side JavaScript. Once WebAssembly breaks that hold, we will look back to the era of JavaScript as a bad, bad dream.

Penultimate means second best, fyi.

Thanks. Fixed.

Because reasons ... First every CRUD app has specific differences in: the UI, authorization system, rules,

It is not taught enough in classic computer science classes (what I did) to reuse existing patterns/RAD/frameworks

Also it is to an extend automated: ui libraries, widgets, SQL/NOSQL, admin interfaces etc are existing and could be viewed, as automation.

because the real world sucks and it's full of chaos, exceptions and special cases.

This was possible in Smalltalk environments where everything was in one "garden". Products like Cincom VisualWorks where doing this 20 years ago.

The web is not unified like that, the foundation is way too shaky. Visual UI builders have failed idea because they cannot encode complex business requirements.

I dunno, mate. Your proposed solution sounds fairly custom to me.

There are lots of tools that do that. But if a tool automatically programs for you, then are you a programmer, or a user?

Programmers do not want to be accused of being users. So they stay away from those tools.

(FYI I am a programmer.)

The Django admin comes pretty close.

because every data model is different, every process flow is different, every UI is different.

Humanity has ever been known for homogeneity. Different people think differently about how to solve problems.

Because management is ignorant of technical costs. At the end of the day, the person who cuts checks says “I want rounded corners with drop shadows, and I want them pink and sparkly.” For example, we already have standard input controls but it’s de facto industry practice to implement custom ones to please those that have no understanding of these invisible costs.

because WYSIWYG went out of fashion.

also web frameworks are not honest about being framework for building what's already out in the wild, they are flexible enough to build the next killer app but not simple enough to create Yet Another Web App.

It's interesting isn't it, we wouldn't dream of hand editing an image file or CAD drawing yet we are happy to use a text editor to change the colour of some text on a web page.

Personally I think web development has gone completely off the rails, I'm glad I don't have to do very much of it.

I would like to quit web development and come back if a modern solution comes along.

We need a cocoa of the web, there was cappucino but it never took off, but I think with web assembly anything is possible, i'd rather use opengl and shaders than html and css.

I even think there's a missing file in between html and css, there should be a data structure that automatically becomes a user interface and another language to for layout, so the the concerns are separated between developer and designer.

This is something I've long believed. A Data Style Sheet to go along with the raw HTML and CSS.

You could even make it as designer-friendly as JQuery while still building modern isometric SPAs as output.

For example:

    #users { from: store.users |where roles.includes('Admin')}

Cascading Data Sheet? that it exactly what i'd want. I saw yaml based templating like slim or haml but then thought it was such a shame to see 'div' or 'b', etc in there.

Yep, it would even interpret the designer's dummy data as a repeating template. If it couldn't figure out the mapping CDS could give hints as appropriate.

The best part is it could be compiled into a component like JSX but wouldn't prevent the page from being previewed with no programming implemented at all.

The template would basically be an HTML mockup.

yep, a fully designed dummy just gets manipulated without any code being added to it at all.

then there's no chance to get in each other's way, as long as the designer names the classes correctly.

As long as they name them consistently really, you would just need to provide custom transform/pattern rules if they could not be detected.

I might work out a prototype of this once I get my framework off the ground.

> Why Isn't the Construction of CRUD Web UI's 90% Automated?

My guess is because web coders have very, very short memory and couldn't be bothered with learning whatever programmers were doing in '90s and '00, both on server side and on desktop side.

> There have of course been hundreds of UI builders. Why aren't they used?

There was this Borland Delphi thing, and these different GUI toolkits like Motif, Tk, Qt, GTK+, wxWidgets, or Java Swing, but then desktop software became mostly irrelevant, replaced with web. Somehow very few people cared to port good ideas from desktops.

> Behavior should be mostly auto generated config from a standardized format such as json schema or a db schema.

There you go with learning history. There were attempts on desktops at doing such things, but they died off. They weren't that good of idea.

Thanks :-)

> There you go with learning history. There were attempts on desktops at doing such things, but they died off. They weren't that good of idea.

What are some examples of tools that did this and what were the issues?

Code generated from schemas or diagrams? All the UML things, or code generated from database tables.

There was an old java attempt that tried to generate the UI automatically from classes - Naked Objects.

It looks like it's morphed into Apache Isis http://isis.apache.org/

I programmed in a DOS tool called 'Magic'. It basically was a screen designer which could work on tables. All code was held in the database, all screens were datadriven (declarative). It ran on the Novell Btrieve database. It was from an Israelian company.

It worked perfectly. Even my brother, who wasn't a programmer at all, was able to write his business software with it. It was reasonably fast. Downsides: theming wasn't possible so all applications looked the same, and Btrieve wasn't as reliable as it needed to be.

But otherwise it was great. I never understood why such a program doesn't exist these days. Tempted to write one myself....

Edit: they still exist: magicsoftware.com.

I've love to see more character cell intefaces. Not everyone wants to run a web/GUI stack for a simple CRUD form. TUIs are often fast, and low requirements. I've considered the same myself, as no modern languages include a real TUI library (no, curses support != a TUI).

There ought to be an open-source Linux-compatible Borland Turbo Vision clone.

Your clients could be $10 Raspberry Pi Zeros.

I think that's what the web community misses. We have 50 years of Moore's Law to benefit from, yet we are making slower applications than ever. The larger these codebases, the more complex they are to maintain, secure, and debug. They need larger hardware to provide the same relative performance.

I want to see more lean, tiny, fast applications. Low TOC, low investment because they aren't complex. Low end hardware that can be made reliable, small, and maintainable.

I think it's hideous that most business applications I see (CRUD interfaces) are 90+% text, and yet they are built on layer and layers of GUI and web interfaces.

Your Raspberry Pi example would be lean and easy to maintain. No virus scanners, no 20GB OS footprint to maintain. No worries about your user visiting Facebook all day instead of working with your business application.

I want applications that are so incredibly dull that they will be taken for granted as much as office telephone.

I've been down this path before. There is R:Base (incredibly hard to Google http://www.rbase.com), which is basically the same. Another one is Wavemaker, for the web.

The difference is that if you're a small shop, the licensing costs will kill you (although Magic was a $2000 product, which is around $3,700 adjusted for inflation. R:Base and Magic cost around the same.

By far the best was the dBase family, mainly foxpro.

Although I never got deeply into the field, I think the UI builders also had the problem that web frameworks do: which is though they make it easy to do the easy stuff, they eventually make everything harder.

That isn't supposed matter because most real-world customers only have simple needs, so you want to bang out the apps quickly. But in practice I have never seen a use case that was really as simple as it was supposed to be -- they all end up in the "too hard for the framework" bucket.

We definitely need better tools as an industry. A good tool would allow custom code to override the framework and the imported components.

XAML from Microsoft and XUL from Firefox for doing UI have sort of died, haven't they? Maybe because they weren't perfect for all use cases?

I still use XAML daily to build iOS and Android apps. It's perfect for 95% of use cases.

Reply on news.ycombinator.com