Software Development at 1 Hz
medium.com
medium.com
I eventually started keeping a journal of every code change I made, along with the hash of the binary that it created. I could use this to make several independent changes and run them all at the same time. When one run finished, I would verify its behavior, look at the binary's hash, and then know that my code change was safe. It was a very slow process, but effective. After doing this kind of development for a while I'm weary of claims like this that imply that feedback with a latency <10s is actually a good thing. High latency feedback forces you to be more methodical in your development and think about the changes you're making.
So does not having tests.
This sounds depressing.
Definitely true on the web side of things. Bootstrap+React+Angular+jQuery+whatever else, minify, deploy on docker, done.
I'm not all in on what I call "Poirot's doctrines," but there is almost always room for method. I grant that some methods aren't needed and speed is always of high value.
However, speed of bouncing from beep to beep in your tools are not always faster than stepping back and thinking.
As an example, no amount of fast feedback will help you complete all of the Euler problems.
I find this one almost a perfect counterexample. The Euler problems are almost ideally suited to high levels of interactivity, in order to tease out patterns and solutions.
By no means am I a massive Euler user, but when I was solving #555 I was sure glad to be using Python. If I had a >1m recompile time I probably would have just given up.
Also... I don't see how any single Euler problem could have minute plus compiles. No matter the language. :(
I also wasn't saying that any language would get long compile times on Euler questions -- they're far too short for that -- but that theoretically if they did it would cause problems.
It is worth noting that many of those problems originate from people that did them by hand. :)
Again, there are no panaceas.
This! People rely on their fancy REPLs and super fast feedback loops and 1000 unit tests too much these days. What do you actually do when you can't run the code? What if you have to debug it just by reading it?
There's a lot to be said about being efficient with trivial changes vs being methodical and able to solve much complicated problems when they arise.
Instead, I believe in thinking carefully about the problem. Close your eyes and visualise the program and its data and control flow in your mind, then write the code. Use a whiteboard or even pencil and paper to collect your thoughts and get a good mental model of what you're trying to accomplish. Block out all other distractions and focus on the problem.
Many others I've talked to are in disbelief when I tell them I can spend an hour writing several hundred lines of code that compiles and works flawlessly the first time, but this is what careful thought will allow. Even with a very fast feedback loop you may spend several times longer fiddling with the code until you get something that seems to work, but actually doesn't in all cases precisely because you did not ever think about those cases while you were fiddling with it and had your attention focused on getting that next dose of feedback.
>>"...also happens if you give them a debugger..."
1). I assume you can cite no research supporting the idea that new programmers are better off with your recommendations?
2). Your idea doesn't seem to take into account that different people think in different ways. I believe this approach was good for you. But as far as we know you could be in the minority right?
For these reasons I don't think there is enough data to make blanket recommendations against IDEs and debuggers.
One could just as easily hypothesize that these tools let you avoid thinking in the small, and help you form a big picture overview that would otherwise be difficult to understand.
I went quite a while with no tools other than a hex editor to type in op codes. I don't think it did anything except hurt productivity.
Maybe you learn or work better that way. I don't. And I don't see how you justify assuming all new programmers would.
Would I recommend an IDE for a low level language like C? Probably not, because it forces a kind of laziness on the programmer.
Maybe an IDE isn't the solution, but a starting point to build upon. Something that's an interactive environment like LightTable has, where you can quickly eval blocks of code and see the end result without having re-compile your entire program. Certain languages are better suited to this, and certain paradigms (reactive programming comes to mind).
Sometimes, exploring the problem space can give you a fuller understanding of a problem faster by forcing you to confront pitfalls that may not be obvious until you try a solution. We use all kinds of wonderful terms for this - "Agile", "Prototyping", etc.
Both extremes - fetishizing planning and up front design, or fetishizing short term iteration and poking things without deeper thought - have their problems, and occur too often. Neither tool is a panacea, but both have their place.
I'm not saying that there are no unjustifiable over engineered java libraries, but the current hype cycle of web frameworks seem to indicate the burden of proof of good design should lie with current technologies as well as previous.
"Everyone uses it" or "it's producing value" doesn't mean it's not an overengineered mess that everyone recognizes as such - it just means that imperfect code still trumps no code. Switching languages usually means tossing out your old codebase, leaving you at "no code".
I have worked on such messes, created such messes (oops!), and cleaned up such messes.
That said, I'm sure there is a Java project out there which actually benefits from stereotypical levels of Java abstraction and patterns - and I'm sure there's a few codebases out there where "my" and "others" opinions differ exactly as you say.
> I'm not saying that there are no unjustifiable over engineered java libraries, but the current hype cycle of web frameworks seem to indicate the burden of proof of good design should lie with current technologies as well as previous.
100% agreed - not that I'm qualified enough at web dev to have much of an opinion on this. If anything, the churn of web frameworks smacks of being both overengineered (do you really need a whole framework for that?) and underengineered (wait why are we replacing things yet again?) simultaneously.
But it's the exception rather than the rule. Once you have something like Spring in your codebase, to take care of modularity and reuse, everything else should be coded with as little "abstraction and patterns" as possible.
It's good that you mentioned Java, because it is a language which I find extremely IDE-centric, and I suspect that's also what causes easy premature generalisation --- creating new classes with tons of boilerplate automatically generated by the IDE is so easy that it encourages programmers to. That doesn't help one bit with the details of the algorithm, unfortunately; it often gets "OOP-ified" into a dozen classes and much-too-short methods created as a result of the "fiddle with it until it works" mentality.
Of course it's possible to push code around on the page until it seems to work, just as it's possible to push symbols around the page until it seems to work when answering a mathematical question on paper. (Unfortunately some languages/compilers will run code that doesn't make any sense, but that's more true on paper, not less)
> Instead, I believe in thinking carefully about the problem. Close your eyes and visualise the program and its data and control flow in your mind, then write the code. Use a whiteboard or even pencil and paper to collect your thoughts and get a good mental model of what you're trying to accomplish. Block out all other distractions and focus on the problem.
It's funny how many problems I've solved by writing code on paper/whiteboard when I got stuck doing actual programming. It's so much easier to focus on the problem when there's no code to run.
Another thing I found useful is just reading the code outside of an editor. Either by printing it out and scribbling over it with a pencil, or just reading it on a phone/tablet that can't run the code.
> Many others I've talked to are in disbelief when I tell them I can spend an hour writing several hundred lines of code that compiles and works flawlessly the first time, but this is what careful thought will allow.
I've been having the same experience. Recently at a uni we were given an assignment to write an interpreter for a rather simple imperative language (conditionals, loops, simple recursive functions and stack depth checking). We were given 3 hours to write a program that could interpret a sample program.
Most people struggled to get anything working at all during that time, since each and every one of them I talked to didn't have a clear picture of they were trying to build.
It took me a little over an hour to write the whole thing in almost a single pass, in a modular fashion with separate tokenizer, parser and evaluator with the necessary checks. There was no need to run the code, most of it was rather trivial, implementing simple state machines. It was quite a bit of code (over 1000 lines), but there was almost no thinking required if you knew how the parse tree should look.
In situations like this I'd even say it's hard to make the program not work if you're methodical, working step by step and checking if you've covered all the cases.
But in general I agree with you.
But I can also adopt a much more interactive, experimental approach in situations where experiments are cheap and easy.
It all depends on the nature of the task.
If you think small code/functions are the just the result of "short term decision", that is wrong. Small and short doesn't imply ill-developed, short-sighted nor a lack of whole picture.
Small and well-thought code often contains an abstraction. And compiling such an abstract-level code takes even shorter time, still do some syntax check and (optionally) checks types. If you fix the abstract layer then you proceed to the details.
"Close your eyes and visualize the program"? Why not just draw the image as an abstract program on the screen? You know, Hackers and Painters is a real thing. Good abstract code describes itself on the screen, you don't have to imagine the behavior in your head. Code is much better than your volatile image in the head that may go away if you go to sleep.
There is also an attitude in computer music that low latency audio and realtime systems are simply part of the march of historical progress and ultimately represent a step forward from the days when composers could wait a week to hear 15 seconds of music rendered.
I found that I like the think-write-render-listen-repeat loop that comes from a compiled music workflow. (I discovered this when writing a -- initially very slow -- nonrealtime python computer music system which in the early days would sometimes have me waiting a few hours to render a couple minutes of music.)
Realtime computer systems enabled all sorts of new forms of live improvisation, interactive algorithmic systems, etc but sometimes it's very nice to have to sit in a quiet room with a text editor or notebook and think your way through the next long render.
When what you're doing is simple but error prone, or not well defined fast feedback can make you much more efficient. If you're using an underdocumented API/dataset, the best way to understand it is to probe it with code, quickly iterating.
I used to write a lot of LaTeX and make lots of simple mistakes (missing backslashes typically). I found having an environment where I found my mistakes as soon as I made them more efficient than searching through the output for all mistakes, and then for each mistake finding the corresponding source and correcting them.
In the end you can write code to be read (aka maintainable), to be run (aka fast), or to be written (aka cheap).
PS: Sure, in theory it's simply better. However, people get lazy make some change and assume if it's passing their tests it works.
That just means the problem is deeper than your understanding of it.
When I'm debugging, I'll kick off several variants to test different ideas, and write myself notes on what I expect to learn from each one when it finishes. Then I can go off and work on something else until it's done, and use the notes to get myself back into the debugging groove. Without the notes, I'd find myself an hour later struggling to remember why I even ran a particular variant.
In my new team I got it from 30s to 5s and the effects have been amazing. I have to thank VScode and Gulp to make this possible. With typescript vscode does fast parsing of your code on every keystroke and gives squigglys instantly. On every save, gulp does its magic and vscode runs the problem matcher and shows more squiggly on my editor.
The browser knows when a file has changed and refreshes immediately. Vscode has a chrome-debug plugin so my breakpoints hit immediately in the IDE.
It's been an amazing experience so far. We've also revved up our CI systems to run 1000's of tests in parallel.
You actually enjoy work when you don't spend forever waiting for things to compile.
I was proud of being part of the team that got the compile time for the core of our product from about 2 hours down to about 30 minutes (the same amount of code, loaded into a RAMdisk and built on a 16-core machine builds in about 2-3 minutes today, but most of our full builds take from 20 minutes to 2 hours now).
After you build the core, a fairly simple plug-in on a reasonable development machine will compile+link in about 5 seconds, maybe. How long it takes to test the fix depends on how far into the program's operation a failure is expected to occur...anywhere from a couple seconds in, if it can't contact the server, to an unbounded length of time, if there's a rare bug triggered by some weird pattern in the customer's data or use of the system.
I enjoy hearing about what other people expect from their dev environments. I've worked almost exclusively in large C++ systems.
If the need of better tooling is clear, the OP does not mention one important point: faster machines have created some sort of fast food approach to programming, where programs are built on the go with the help of the appropriate tools (some sort of computer-aided programming).
Back in the 60s, when programs where being shipped by snail mail to some data center somewhere in the country to be entered by a random assistant and scheduled in a long pool of jobs, turnover was counted in days. Yet, we walked on the moon.
So beyond better tooling, maybe a key to maintaining that attention span is check is to simply spend more time on the design board.
Observe Google and their management of Android.
Again and again they have pushed out some half-baked X.0 version with the unstated intent that it will be sorted in a near future update. A clear example of that was the introduction of the Storage Access Framework.
This is the web mentality seeping into the development of firmware and physical products, and it is leading to a massive culture clash and crap products.
Basically you can't just code, push, run, repeat on devices in peoples pockets and bags like you can a web site.
Or how at least every two days, the phone's wifi and LTE connections just stop working, for no apparent reason. But if I Force Stop on Google Play Services, they suddenly work again. This has been going on for several months now.
Of course, these issues didn't used to happen with older versions of Play Services. And if I downgrade the Play Services app to the version installed with my ROM, I never have these problems--but then I can't use any current Google apps, like Gmail, Maps, etc.
The AOSP software is mostly fine. But the Play Services side of Google simply cannot be trusted. Their CADT/web-style of development is almost enough to push me back to iOS.
I want a phone that just runs Debian. :(
Reducing the turnaround is a good thing, because that time that would have gone into triple-checking variable names can now go into triple-checking the algorithm on a whiteboard.
This summer I worked on an application where it took 8 minutes to see the effect of changed code. Between each iteration, I had forgot most of the assumptions I had made. So when things didn't work, I had a hard time figuring out why my model of the world didn't work. I basically started from scratch each iteration.
Possibly very obvious, but it was interesting to learn how I attack a problem. And scary to see that I didn't just lose the 8 minutes between each run of the program, but much, much more.
(My point is: debugging first in your head is yet another skill that should really be taught to everybody but isn't).
Along with smelting iron. I don't know why people advocate skills one doesn't need, saying "but they make you better".
Skills you don't need will atrophy, because you don't need them. Conversely, skills you need with strengthen. Why would I debug first in my inaccurate head when I have a perfectly accurate debugger right here, and I can run my test suite in ten seconds?
Debugging first in your head is a good skill if actual debugging will take hours. If feedback is instant, you're probably faster writing the first thing that comes to you and iterating on it.
Similarly, thinking about things before you do them will almost always be something you could just skip out on. But... It can be very helpful. And exercise is a great way to get better at work. "Practise makes perfect" and all.
And, I should be clear, I was saying "exercise" to refer to gym style exercise.
So I understand the concept from the point of view of task-switching.
But a lot of people in the comments allude to needing a feedback loop for each and every line of code, which sounds absolutely horrifying to me - is that really what development is like in some environments? you can't really be sure of what a single line of code is going to do?
This guy seems like and sounds like a serious developer, so I'm totally confused by this statement.
Dynamic languages that don't do compile time type checking are not toys.
I used to only write in Java or C++, but I think it's a stage of maturity as a developer to realize that you can develop code that can take arguments with the assumption that the objects sent in are of types that will have the behavior you need to work with them.
If you argue that the code is faster when it is compiled- that's fine, and I agree, and that's good, if it matters.
If you argue that you need types because otherwise you can't be safe, I'm sorry, but that's like being a helicopter-parent. Sometimes maybe you can't trust what is calling your code even when you give it trust, and that's valid; just like as a parent, sometimes the child really needs that level of micromanagement. But, for a lot of if not most of practical web development, you can use dynamic typing, and most children do not need that level of micromanagement.
There's nothing wrong with languages that provide type checking, but it isn't necessarily a deficiency when it's not there.
I use TypeScript because that level of "micromanagement" saves me more time in avoiding bugs than I spend adding static annotation. Hell, the improved intellisense alone means I no longer need to read docs in a lot of cases, meaning writing code is faster for me too - even ignoring bug rates. (Of course, it's sometimes prudent to check the docs for things like edge cases regardless.)
In C++-land, I've started using e.g. clang's threading annotations to good effect in catching some of the most heinous bugs to debug - incorrect multithreaded code that forgets to do simple things like lock mutexes meant to protect data structures.
I've dabbled in a toy project in Rust-land. The ability to catch and prevent data races is fascinating, and the ability to stem the tide of null dereferences at runtime seems pretty handy. Have you never had a hell-to-reproduce null deref that only occurs in your release builds? It's pretty bad when it ships to a large number of customers.
I see SQL injection vulnerabilities, and wish APIs properly segregated SQL Data from SQL Commands - two entirely different types of things entirely.
And yet for awhile I was a lot more forgiving of dynamically typed languages. Until I was able to compare JavaScript vs TypeScript - which I'd argue started as basically JavaScript with static typing tacked on as, effectively, an afterthought.
> If you argue that you need types because otherwise you can't be safe, I'm sorry, but that's like being a helicopter-parent.
If helicopter parents were as beneficial as static typing, I'd have a lot less against them.
This is backwards. Working without types is like walking around with your eyes closed: sure, you can do it, most of the time; if you're not doing anything particularly dangerous you can even do it reasonably safely. But it makes everything a lot slower.
The arguments against types usually boil down to either, as the saying goes "The belief that you can't explain to a computer why your code works, but you can keep track of it all in your head", or having only used languages where the explanation to the computer is so cumbersome as to not be worth doing (valid, but only in the scope of those languages, and the correct response is almost always to get a better language).
Try using a language with a decent type system some time (something along the lines of OCaml, Haskell, F# or Scala). Back when I'd only written Java and C++ I also thought type checking wasn't worth it.
For me the most difficult argument against static types is that the sweet spot remains elusive: some type systems are too simplistic (e.g. the difficulty of writing generic print in OCaml) while some are too fancy and difficult (e.g. how many people understand even most of GHC Haskell's type system?).
There's also some real problems with compiler error messages. A great type checker needs to be able to explain problems understandably, or decoding the type errors will be more difficult than tracking down a null pointer in an interactive debugger.
I wonder about the possibility of making type checkers more interactive. It can be hard to understand them because they build up lots of implicit understanding that's not apparent.
I don't advocate a full type-safe language. In Common Lisp I generally don't have to declare any types. The compiler points out obvious, unavoidable type problems that it can deduct, e.g. inside a function that has some typed things such as literals. I can then add types as I like, to both variables and function interfaces, and the compiler points out more.
Another property of Common Lisp is that if you add type declarations they speed up your code if you compile with speed==high and safety==low. But you can also compile your code with safety set higher than speed. In that case a compiler like SBCL turns your declarations into runtime type assertions.
Then you run your automated tests in both modes and you have higher confidence in the code.
- a super-fast assembler
- a super-fast way to get the assembled code over to a target system
... and my turnaround time was on the order of five seconds: Edit, hit a button making the target ready to receive the code, assemble: Running.
It almost didn't matter that I was writing 6502 assembly; things just fell together and it was magic.
Years later I was in a place where it was common to have half-day builds (30 minutes if you arranged things well). In fact, my last three weeks on that job I never got a working build at all, despite our group having an entire source control team whose job it was to make builds work.
Current big project, it's about 30 seconds of tool churning, then another 30 of startup time. Could be better.
- Burning EPROMs for the hardware bring-up of a consumer 68000 machine. Download and burn time for six EPROMs was well over 45 minutes. Things got better when we wrote a downloader, but initial bring-up was kind of painful for a couple of months.
- An unbelievably crappy and development-hostile environment for set-top boxes. Getting 900K of code to a device could take 20-30 minutes, and the transfers often just abjectly failed. "Working as designed" said the people running the head-end. If you wonder why set-top-box software sucks so hard, this is one of the reasons. The whole TV industry is a fractal of shitty practices. Not that I'm bitter. [I did a work-around that let us do dynamic code loading over a different pipe that we theoretically weren't supposed to use "for network stability purposes" and got a biiig bonus after our team's productivity went way up]
- Game development using a cross-assembler on a minicomputer,. The mini was also used as the department's email and office memo system. 45 minutes during the day turned into less than 5 minutes at night, so you can imagine the hours I kept.
- Working on some Windows internals, the less said about this, the better.
The common thread: When bad decisions and designs were institutionalized, things never got better. When it was possible for individuals to improve things, they did.
I took a look at Wirth first Pascal compiler too, it's just too files, of acceptable length, lot of coupling though, but it's easy to guess that it's an imperative style that brings a lot of mechanical sympathy.
For dynamic languages, use the `watch` command combined with a host script that loads the code you're developing dynamically and executes it with some interesting parameters, and dumps the output. As you edit the source, you can see the effects immediately, character by character.
You can get the same effect in something like Java in an IDE if you can structure the interesting code like a game loop (ideally it is a game loop): it's being continuously evaluated, so every change has an immediate effect. You can see Notch use this technique in this video here:
https://youtu.be/rhN35bGvM8c?t=5757
As he edits code in the IDE, the application responds dynamically.
Here with a more traditionally interactive environment (so updates are applied when you save the method): https://www.youtube.com/watch?v=ArcClqt2vTc
Here in a really live environment I call CodeDraw, with updates computed on every keystroke (the "Run" button is just left-over UI, it doesn't do anything): https://www.youtube.com/watch?v=sypkOhE-ufs
Most interpreted or incrementally compiled systems should be able to do this.
I'm getting this with a language that has a very high performance native compiler. And it's sad not more people use this.
I rely heavily on the IDE reminding me of things, and quick compile/run cycles verifying correctness, rather than trying to think thru a myriad of special cases. Relegated to "whiteboard coding" interview questions, I'm left looking a whole lot worse than I am - not because I don't know it, but because I know enough that thoroughness is painfully slow (without leveraging tools multiplying my skills & speed).
Do you really need to compile/run something like fizzbuzz? What if you're writing code that can't be run, such as when modifying a larger piece of code that won't run until all modifications are made?
Isn't there some value in being able to verify correctness of code just by looking at it for a few seconds? Surely you can overlook some things, but with practice probably a lot less than people think.
I rely heavily on the IDE reminding me of things, and quick compile/run cycles verifying correctness, rather than trying to think thru a myriad of special cases.
The peril of that sort of workflow is that you often won't realise the importance of those special cases until it's too late to change things easily. "It looks like it works, it compiles and runs" --- I've heard this sentiment from such IDE users many times, and yet when I inspect their code, they inevitably missed something important.
Someone once gave me a phrase I like to keep in mind when programming: "How can you tell the machine what to do if you're not even sure how to do it yourself?"
Productivity plumets if a change takes long enough to be visible for me to justify looking at Twitter.
---
Also, I tend to do all my work using a realtime-feedback model like this.
I discovered inotifywait a few years ago, and consider it to be the coolest thing I've discovered. Using a loop like:
while true; do clear; ./program; inotifywait -qq -e delete_self program; done
There are some caveats though.
The biggest is that this approach doesn't work too well for things that need to be killed to be restarted, like socket servers; that's fixable but nontrivial and likely project-specific too.
The second problem are the race conditions that will likely arise between your editor's slow file-save process and inotify's fast response time, producing irritating "File is in use" errors 50% of the time (since inotifywait exited, bash looped, and ./program is trying to be read by your compiler or interpreter while still locked by your editor).
The DELETE_SELF inotify event is specific to the Geany text editor; when Geany saves a file it does quite a few operations, and DELETE_SELF is amongst the last (but inotify doesn't see any of the following events since DELETE_SELF marks that the file - that inotify was watching - got deleted; this is thankfully coincidental with Geany being done with the file). You'll need to do something like "inotifywait -m program" and watch what events occur; inotifywait exits on the first event received; hopefully there's a lone unique event at the end of the sequence. Worst-case scenario you might have to add in "sleep 0.05"; I have not tested the success of prefixing the compilation step with something like "while [ ! -r program ]; do true; done".
If you used up your energy by the time you made it through the simple stuff you are looking at a bad time later.
Ironic when in the past I have worked with C++ code bases that would take 20 minutes to compile, though incremental builds and parallel builds brought that number down.
Also in embedded systems, I've got 3 minute cycles to try something out. However most of the coding is done on a PC with a fast unit testing cycle which dramatically improves feedback loops.
I know rails has something similar but I haven't worked with rails for nearly 2 years now so I can't tell.
As a side note, having a short dev loop is really good for education.
One of the problems is that the things you need to have a proper CI system conflict with fast build times. Proper CI requires a clean checkout, and a total compile fromscratch. If you're doing something in Docker, you ought to start your Docker process from scratch. If you're in a VM, you really ought to revert to a snapshot to make sure you're not accidentally accumulating un-CI'ed state. And so on.
While in normal development you may have a very fast turnaround, a properly configured CI system needs to assume the worst, start from scratch, and build everything, in every combination you support. (You may also want a less accurate CI build that trades speed for accuracy and just does an incremental build of some particular aspect of the system. But that should be supported by the full CI I describe here.)
Consequently, something that fails only 97% into that build process, and only fails on the CI server, can be very annoying to fix. But you don't really have a choice, because any hacky alternative is too risky. A CI system that has human intervention isn't a CI system.
that said, i agree that there should be effort, or at least roadmap, to get CI compile times down to near native
to play devels[sic] advocate we've had lengthy compile times on systems and native apps since time immemorial, it just seems that browser work has caught up in complexity and ubiquity.. even if some of that complexity can be blamed on negligent or lazy development
At work a lot of stuff is like this. Enterprisey applications that take 10, 15, 20 seconds to process each entry. Time enough to get distracted. Time enough to tab over to email or some other application and lose focus. Just overall mentally fatiguing.
Developers who are annoyed by 10 second delays in their development process should remember that their users will be just as annoyed by slow response time in applications. Cut the weight. Make it faster. I'm more and more convinced that nothing else will go as far in making users happy.
Yes I prefer faster loops, but nerver had a job where I could influence the build pipeline...
I was running into some 1-minute link times per project on Android, with several projects. Investigated and found out there was an alternative linker named "gold" - a few minutes to figure out how to reconfigure our builds and our link times were down to ~10 seconds / project or something. Nobody complained when I checked that in ;)
Dealing with C++ build pipelines for large projects, there's only so low I can get build times for the whole project - better to try and make the data hot reloadable in a lot of cases. Similar goal: Fast iteration loops. Even if you can't fix them for the entire project, you may be able to fix them for whatever you're working on right now.