Why is it so hard to write a scaffolding tool? (2019) | Hacker News Reader

Why is it so hard to write a scaffolding tool? (2019)

jfreeman.dev

48 pointsnoworriesnate47 comments

The problem is not the tools. The tools are just fine. The problem is the scope of the resulting scaffolds, and the diversity of their target spaces.

I primarily work in Python, so that is my reference. Django provides its own scaffolding tool, but it is too opinionated. Flask provides no scaffolding. Framework specific tools are rarely sufficient to meet the needs of an organization. Regardless of language and framework, choices must be made, and when you have a team of any size, everyone needs to agree on the same standards. We all know this.

If one limits a scaffold to the lowest common denominator, the base standards of an organization: directory structure, configuration, CI/CD, testing, library management, documentation, linting, packaging, container configuration, i.e., everything that precedes actual functionality, scaffolding is not hard, and is very helpful.

But as soon as one tries to drill down into specific functionality that is not common to all projects, then scaffolding quickly becomes difficult and unwieldy.

IMO, the key is to not think of scaffolding as a tool to speed development, but as a tool to enforce those standards of an organization that are common to all projects in a given language/framework. Resist the temptation to go further. If your scaffold needs more than a half dozen or so inputs, then you are probably expecting too much of it.

Expanding on this.

Scaffolding breaks my mental model for reading code. This often results in more mental overhead and slower/worse outcomes.

When scaffolding generates code for me, I read it as psuedo-blessed code. Rather than starting from a clean slate and working through my mental checklist, I must interpret what the scaffolder created. Figure out what I _actually_ need and figure out what needs changing.

By contrast, without a scaffolder, I feel like it's much easier for me to build a mental model around the problem I'm solving - focusing in on the more important components first. Yes, I know that I'll probably need 20 files to complete a feature. However, I don't need those files right now.

To your point about not using scaffolding as a tool to speed development: I have found that rails' scaffolding commands make me lazy. I will routinely destroy, and recreate, an entire scaffold because I need to change it; somehow I think this is easier than just editing the file I already have on my system -- only to then spend 10 minutes combing through all the changes rails made so I don't lose any of the changes I made.

It's very silly.

... and God granted zoomers homoiconicity, macros and parenthesis.

"No," said the zoomers, "we want to write XML, build scaffolding tools and eat shit."

"Your language will never be a LISP," was the God's reply.

Does it come from somewhere or you just made that up ? I have a really strong feeling of déjà vu but cannot figure out from where.

I just made it up. But I guess we both must be reading the same kind of memes or something.

you may be thinking of rootless root, ESR's zen koan styled thoughts on software http://catb.org/~esr/writings/unix-koans/index.html

I'm clueless. How do these fix the scaffolding problem?

If I start a new Python project I have a short checklist I go through

1. Poetry (dependency management) 2. pyenv and pyvirtualenv (execution environment management)

Just these two are somewhat of a headache to get going. Once they're up, though, they do the job they're supposed to in an admirable fashion.

How would a language with homoiconicity and macros handle this?

I am not too knowledgable about Python to tell you exactly.

But let's take Common Lisp for an example and why I don't feel the need for a scaffolding tool. Well, actually I do need _some_ very basic scaffolding, namely: create two directories called src and tests + add two very short project files with my names and project names embedded in them, along with a couple of package files. Nothing fancy and certainly nothing that's particularly problematic. Everything that's part of the project code (such as package description with imported symbols and such) maybe be done by using a macro from an external library (ex: certain functions of some library that I import in each of my projects).

But you are right anyway, not everything is a language-specific problem here, only the stuff that gets repeated in code, from one project to another. In CL that includes documentation and the testing framework (from the article's points). Well, the build/project description is just a macro too. Anyway, I didn't mean to diminish the major points of the article in any way. The author is right about the fact that generators are not aware of the user's changes and that can be problematic at times (as with his license and license metadata example).

Because scaffolding by itself is an anti-pattern and you shouldn’t do it. The proper way to factor code is libraries/packages, not automated copy pasta.

I tried many times to do what you suggest, and every time it was even harder than the original issue. Maintaining separate packages is tedious (don't forget to commit/push/republish/upgrade after testing) and error-prone version/feature/branch-wise if you can't stop and pay full attention to it every time you change or need it. Not worth it in my book.

Perhaps, if you’re just creating a new application every couple of years. For library authors like myself, a scaffolding tool is exactly what I need to quickly make the ecosystem of small, reusable libraries that you speak of. Do you expect Sindre Sorhus to manually copy the MIT license to each new project? What about the CI config? The .gitignore?

Okay, you might say, Sindre is an exceptional case. But on any new project, you might forget to include the correct set of files or screw up how they are set up by not filling in or updating a copyright notice, project version number, repo link, etc. Sure, you can cut down on the number of files you start your projects with and the fill-in-the-blank values they might need, but that only gets you so far and it reduces the initial usability of your project.

A scaffolding tool helps maintain consistently high quality across projects. For example, my Yeoman generator automatically fetches the latest version of my favorite CLI helpers and testing framework and other dependencies, so I never accidentally start a project with an outdated, potentially insecure codebase.

https://github.com/sholladay/generator-seth

Not everyone needs something this fancy! Another commenter mentioned editor snippets, which is probably what I would use if I couldn’t have Yeoman. But when you make new projects on a regular basis, scaffolding is 100% the way to go.

Like I implied in my reply to GP, this is to cover for deficiencies in the language, tools and environments. If you can't incorporate this common logic by reference, then it's a deficiency of the tool(s). There's no first principles reason why you can't name a common config for ignoring git files, and incorporate it by reference--either git does not have this affordance or you haven't found it. There's no reason you can't apply principles of software reuse to CI configs for similar pipelines: Screwdriver does it, for example. If other CI tools don't, that's a deficiency. Licenses, frankly, are the same; they need to be incorporated as text in releases and distributions, but they could exist as references in whatever you consider your "original source" (and often actually do).[1]

Of course, you don't always get to choose or modify everything about your environment, like how git works, how your CI is configured, etc. But it's not a bad thing to admit that scaffolding is at best a papering-over of deficiencies in these tools, and not a desirable thing in itself.

I'm not familiar with Yeoman, but like other necessary but dirty things we do, we should at least try to encapsulate it and do some implementation hiding. You wouldn't generate language bindings from protobuf and then manually modify them--and the same kind of discipline should hold for these scaffoldings. You should be able to repeatedly regenerate all their artifacts (even if they get checked in) with confidence--and once you've done that, you have effectively removed the "scaffolding" and replaced it with a build tool.

---- [1] As a bit of a tangent, there's an even worse sin lurking here, which is that you have the same fact but need to capture it twice (or more) in different places: in package metadata, however you do that in your language; possibly in project metadata (if however you host your project doesn't discover it); and in the text of the license that must be distributed. If you don't rely on a single source of truth for this, you're asking not just for an ergonomic problem with updating your scaffolding, but an actual, material inconsistency, like when the COPYING file contains a different version of a license than the one that's referred to in your .gemspec. I point this out as a conceptual detail, not because I think it's an actual frequent problem.

> you can't incorporate this common logic by reference, then it's a deficiency of the tool(s).

If that's the case, then I know of zero languages/ecosystems with tools that are not deficient in this manner.

I agree with you entirely, but I have to do so admitting the reality that not every environment or language is so capable. Don't get me wrong, I still think it sucks, because it breaks robust mechanisms of reuse and updating dependencies in favor of very fragile ones. I'm fighting the fight right now to at least capture these dependencies in the form of tooling rather than scaffolding. But when the language is inflexible and really demands deduplication rather than boilerplate, it becomes really difficult to make the case.

The author glances off another problem in the "Incrementality" section. Assume you have a few projects scaffolded. How do you retrofit any updates you've made to the scaffolding to an existing project that has probably diverged from the original vanilla scaffolding in non-trivial ways?

diff3 is generally how the problem of a shared base with two child histories that need to be merged is handled. This works on a per file basis, so if you have moved files or functions around, it gets more complicated.

Was in my todo to investigate scaffolding tools. Was thinking of using:

https://github.com/norskeld/arx

Curious what everyone else uses? Or you still start each project with empty folders and manually add things?

I would love it if there were a massive, correctly-tagged list of starter templates for all kinds of combinations of frameworks/libraries. The question is, how should you manage that list?

One idea I've been implementing for fun lately is a git repository where each branch represents a combination of optional features that can be generated. E.g., the branch `aad/aspnetcore/hotchocolate` is the starter code for an ASP.NET Core GraphQL server that uses Azure Active Directory for authentication. Whenever I make a change in the aad/develop branch, it automatically merges to aad/aspnetcore, aad/aspnetcore/hotchocolate, aad/aspnetcore/rest, etc. The problem is there are a lot of merge conflicts as you can imagine.

I start each project with a README, and then slowly add stuff.

Very few of my projects existing templates closely enough that I feel like I could get away with using a scaffolding tool. When I first used create-react-app, I spent a half day ripping apart the results and adding the parts I liked to a new project, incrementally.

I feel like copying build scripts, configuration, and project structure from existing projects you have experience with is no worse than using a scaffolding tool. Copying isn't a bad thing.

I use [0] maven's archetype plugin [1]. I sense, though, from the other responses and yeoman's generator list [2], my expectations are on the low end. Other than setting up the source package, I mostly have it fill in documentation templates.

[0] https://bitbucket.org/Nzen/zaftig-offal-hamisha/src/master/s... (file templates)

[0] https://bitbucket.org/Nzen/zaftig-offal-hamisha/src/master/s... (plugin config)

[1] https://maven.apache.org/archetype/maven-archetype-plugin/us...

[2] https://yeoman.io/generators/

I maintain a set of editor snippets instead. There is a snippet for each of: main source prolog, package.json, visual component, express app, standard config, ts config, index.html, etc etc. If I need something that I deleted earlier, I just expand a snippet in a new temporary tab and copy relevant parts back. Much easier than maintaining or learning a scaffolding tool.

Encourage everyone interested in this space to take a look at projen, and a proof of concept inspired alternative pdkit.

https://github.com/projen/projen

https://github.com/justinm/pdkit

A design pattern boilerplate generator in different languages like Python and C++ is something I’ve been thinking of lately. It would be super useful for new devs at work too.

Because from a 10,000 foot view, everybody wants the same thing, but when you get down in close it turns out they don't, and the differences dominate the similarities. Common misunderstanding. A very similar root cause to the UML failure story currently on the front page.

Scaffolding tools in particular have the problem that once you union all the different requirements the resulting scaffolding is insane and impossible for anyone to understand. Some people need to have C modules in whatever non-C language you're working in. If your scaffolding doesn't include that they need to do a lot of work. If it does include that, a whole bunch of other projects now have a lot of crap they don't need, which may be difficult to extract properly from the resulting scaffold. Multiply this process by dozens of features and you end up with something that is basically guaranteed to be a monstrosity that nobody can understand, to be missing critical features, or most excitingly, both at the same time.

The natural solution many will jump to then is "Oh, we'll make it configurable!" Now the scaffolding authors are responsible for testing the exponentially-growing state space of possible configurations, and the scaffolding, whose whole point in life is to be simple, is now complicated. And most of the time, if you discover your scaffolding configuration was wrong after you've already started work it's non-trivially challenging to fix the config and re-run. This basically turns the scaffolding into a non-Turing complete domain-specific language. Those have their uses, but "operating in a fundamentally Turing-complete programming space" isn't one of them. It's also arguably an inner platform, replicating what the language already has.

But jerf, you protest too much. You seem to have disproved the utility of scaffolding in general, but I know that I've used it before successfully!

Yes, you can solve all these problems with basically the Gordian knot solution. You just provide scaffolding anyhow, and it can be helpful. But my proposition here would be that you basically just accept that any given scaffolding can't be The Solution For Everyone, you don't go crazy trying to provide endless configuration, and you just accept that it'll be missing features for some people and have unnecessary features for others. There is absolutely value in simple scaffolding even despite what I say here, especially for things like "Here's how to start getting into Ruby on Rails!" or other similar frameworks where it honestly may not be practical to expect someone to be able to bring up a new project from an empty directory. My point here is more to not jump into the Turing tarpit of spending tons of time and resources trying to 100% solve this problem everywhere and for everybody. Grab some quick wins, be aware of the risks, be aware of the infinite regress of feature requests that lies down this path with ever diminishing returns, and move on in life. There's plenty of other situations like this in the programming (and engineering in general) world.

Aside from scaling issues inherent in Ruby, Ruby on Rails has got to be the most successful scaffolding framework in terms of high compatibility with a variety of project ideas.

As much as I hate monkeypatching as a design principle... and I do!... it does make it much more effective to have a highly extensible scaffolding that doesn't have to include everything in the scaffolding itself, but can plug in very sophisticated things to the base scaffolding later with a simple import statement.

My personal opinion is that the disadvantages of this approach still far, far outweigh the advantages... but let it not be said I don't recognize that there are some advantages to it even so. :)

This makes me think there is something in the abstraction that begins to hint at an elegant solution.

What about monkeypatching makes Rails tolerable? Why do "the disadvantages of this approach still far, far outweigh the advantages"?

Instead of a one-size-fits-all tool, I prefer having a ton of sample projects which demonstrate different use cases. Give me the barebones hello world, the typical simple project, and the nightmare of C dependencies, and I can figure out what I need.

That's not trivial to maintain, but neither is the tool.

Hmmm interested to see how ML might fair here. It sounds like copilot is successfully generating code.

The combinatorics problem with "options" on generators makes it pretty impossible. I'm sure we've all search for something like "typescript react Apollo jss boilerplate" and crossed our fingers that it was updated in the last year. Imagine changing just 1 of those things and how many code decisions change.

In almost every company where I work with (with the exception of big companies) I am always the go to person for JS tooling related.

I don't want to do that work but I guess the knowledge compounds, knowledge about intricacies of various JS tools and how they relate to one another, and no one else want to do it.

The amount of customization that can happen in JS world makes scaffolding tool really hard.

See https://news.ycombinator.com/item?id=33085507

I have a horse in this race, with https://apibakery.com/ (API service generator for Python and Node).

I think today's scaffolding tools (eg. cookiecutter) are just a fraction of what they could be. As the article points out (at least that's my reading), there's tension between expresivness of the tool and ease of use. Incrementality and composition are two ways of trying to handle that.

My approach is interactivity, using a GUI instead of command-line tool, and instant preview. Within a graphical UI, the user can more easily grasp much more complex configuration scenarios and explore options, and they get instant feedback on how that will look in practice. My bet is that this will allow adding a lot more options while not bogging down the user the way the command-line tools have to.

I've spent a considerable amount of time with buildroot, a "scaffolding" tool for Linux operating system images. Buildroot has some assumptions that make the composition of arbitrary files in a filesystem easy. First, system images are assumed to be static once composed. Specifically, you aren't meant to have a package manager, or run updates on any software. Persistent changes are fine for non RAMFS systems (persisting changes for a RAMFS is a fun puzzle to solve). Second, when you update a system image, it is a complete image by itself. Updates are not incremental.

From the scaffolding perspective, this means that every time you want to change something in the scaffolding, you rebuild the entire thing. The Linux kernel follows a similar tack; if you make changes you rebuild the entire thing.

Egh, I will just link to my comment from that other thread: https://news.ycombinator.com/item?id=33083868

We need something like a "gradually typed version control system".

Have not used any of this tools other than in toy examples but I'm thinking the biggest problem is to keep the code and the template in sync.

I know I'm asking too much but I wish I can just update the template and have all the files generated by it updated too.

Check out my project https://docs.hofstadter.io

This was a core tenant of the code generation. Additionally, you can also edit the output fields and still keep support for regeneration after template updates.

Btw, since this is a scaffolding thread, does anyone know a tool which could create files and folders in cwd from a single-file template? Something like

  /folder/subfolder/
  /file.txt
      some text
  /some.sh +x
      #!/bin/sh
      …

I could use a zip/tar file, but repacking it on every change would be cumbersome. There must be something simpler.

Brainstorming: a Makefile or script with embedded file contents using heredoc literals.

> machine-editable configuration files. I'm not just talking about parsing and pretty printing; I'm talking about preserving comments.

Augeas is your guy here.

https://augeas.net/

I think scaffolding is YAGNI. I've only needed it when I didn't know what I was doing.

Lol, yes exactly.. but "most people don't know what they are doing" is probably also your opinion so scaffold tools would be a huge lever for "most people" no?

Interesting article. Felt strange to read through the entire thing and not see any mention of Rails, which I have to imagine is the most sophisticated, or at least the most widely used, example of the genre.

Note sure, but npm create-* might be more used now

It would be an interesting point to bring up, especially on the topic of why they are difficult to build. No doubt Rails' scaffolding benefits quite a bit from how opinionated the framework is about project structure.

Skaffold is my new favorite scaffolding. Language agnostic too!

Self-plug, we are releasing a composable scaffolding command in our next version, release candidate available already

https://docs.hofstadter.io/getting-started/create/

Reply on news.ycombinator.com