Programming Bottom-Up (1993)

I use it as a bookmark aggregator, since I can submit things and leave myself notes. The first account you create will become an admin, and any future accounts are regular users. Passwords are bcrypted rather than sha1’d. Windows works too, but for some reason the repl seems to block all other threads from making progress, so Microsoft users will have to run it with DEV=0 to disable the news.arc repl on startup.

It’s not ready to show yet in general, but there are a lot of advances. My design goal is that if pg ever sees it, he’d want to use it himself.

It incorporates some ideas from Bel, too. I’ll be doing a big write up of all the changes.

The biggest advance from a language standpoint is probably keyword arguments. If there’s any interest, I’ll go into more detail.

The biggest advance from a usability standpoint is that arc is now a thin wrapper over racket. There’s no FFI and no need to translate anything. Arc lists are racket lists. Ditto for hash tables and everything else. You can write racket code in arc by prefixing code #'(like this), which the arc compiler turns into a racket expression (like this). So #'(require (rename-in racket/system [system racket-system])) will do what you’d expect it to do in racket. You can mix racket code and arc in two ways: (#'foo bar baz) will compile bar and baz as arc expressions, but will call foo in functional position. So (#'begin0 (+ "x" 'y) 'z) will give "xy", because begin0 is a racket special form that returns its leftmost expression.

The other way is an equivalent of quasiquoting: #`(let ((x #,(obj a: 1))) x) will return a hash table with ‘a set to 1.

The unit tests in test.arc are probably the best way to see all the language features, but it’s hard to read at times. (The writeup will introduce each concept in a bel-like fashion, but it’s not ready yet.)

news.arc is where most of the power is on display. Making a new endpoint is extremely easy. And prompt.arc shows how you can write a dynamic application without using databases or needing to store any state. The state is in the closures.

As a consequence, I personally practice bottom-up very sparsely and carefully.

I'm glad he acknowledged that library functions is the majority implementation of "bottom up design". In that case extracting primitive api functions & layers composed of these functions is effective at reusing logic & creating appropriate levels of abstraction.

That being said, many frameworks do not offer these layers of abstraction so create a sort of "complexity lock in"...where one has to use the entire framework in order to use a few useful parts of the framework. I find this complexity lock in to be unfortunate as it not only couples the programmer to the entire framework, even if the majority of the framework is not useful. It also creates noise in the conversation, making it more difficult to search for the appropriate solutions at the appropriate level of abstraction.

There is a cost to integrating other libraries into the framework.

Frameworks also tend to become an institution in search of increasing market share. This means there are changes done to the framework. Some of these changes are breaking changes that don't necessarily improve the core development experience in meaningful ways.

And use auto tests to verify functionality. Having a test program running a full end-to-end req/rpy session checking all functionality makes the bug count approach zero.

I am maintaining large scale mission critical C++ applications using this approach. With zero production bugs for 5+ years.

And yes of course split the implementation into reusable parts as much possible. That is basic Software/System Engineering.

Claims that it can't be done in other languages than Lisp is absolutely BS. Linux (for example) is mostly a huge collection of reusable components written in C.

It works for any language/environment and is therefore a more fundamental technique.

Programming Bottom-Up (1993) - https://news.ycombinator.com/item?id=16933030 - April 2018 (64 comments)

Code you re-use is often ready-made bottom pieces.

Some code you reuse calls your code, like frameworks. They are middle.

You might be writing a plugin or extension to an application; then the top is decided and you're making some middle or bottom thing.

[1] https://history.nasa.gov/rogersrep/v2appf.htm

At the outset let me say that I know that Graham highly values Lisp's ability to transform itself into the language you need, especially by using macros. And that's nice, too, but it wasn't what excited me. For me, it is summed up in this quote:

> As you're writing a program you may think "I wish Lisp had such-and-such an operator." So you go and write it

I remember mentally screaming "Yes! Yes!" when I read this the first time, and wishing I could explain it to some of my coworkers. But who could possibly disagree with such a simple statement? When I read it, why did I think it would be hard to explain?

Thinking back, the conflict was between two perspectives on designing the modules or classes of a codebase: one says that you are implementing a Platonic ideal of the domain, another says you are implementing operations that support the functionality of the system.

In the eyes of many of my coworkers at the time, good domain code would have no traces of the specific needs of the program it was written for. It would have its own logical consistency, dictated by the reality of the domain. A Car class should behave like a car, and it should serve the needs of any future car-related programs our company wrote.

This was obviously an incomplete picture. For example, a Car class in a ride-share system probably doesn't have a coefficient of drag, even though a real car does. But some people felt that the naive, incomplete picture captured such a profound and helpful ideal that it was better than a complete picture that acknowledged the demands of the program (kind of like it might be better to tell a four year old "lying is bad" rather than tell them that it's super complicated and mommy and daddy lie all the time but only in the right ways for the right reasons.)

I hated the way this ideal glossed over the obvious truth that the supposedly neutral domain model was in fact very selective. Any Car class would specify a tiny and very specific subset of the behavior of a real car, and it would do so at a specific level of fidelity, which needed to support the functionality of the larger system. Because I hated pretending that we weren't writing the low-level code for specific high-level needs, I found it refreshing that Paul Graham would so clearly put the needs of the program first. He wrote domain functionality on demand! And if he left coherence and maintainability of the low-level code as an unspoken exercise for the reader, that was no worse (and better in my mind) than leaving unspoken the connection between high-level functionality and low-level design.

Anyway, I was always confused that Graham called this "bottom-up programming," because he explicitly goes top-down. He discovers a need at a higher level ("As you're writing a program you may think 'I wish Lisp had such-and-such an operator'") and then he drops down to the lower level to provide it ("So you go and write it"). To me this is opposite of the approach that says you start at the bottom and implement a Platonic ideal of the domain, without first explicitly considering the needs of the program.