3 lines of code shouldn't take all day
devtails.xyz
devtails.xyz
In other words, reducing the iteration time reduces the motivation to get it right the first time, and the associated deep reasoning/"mental execution" skills which are required to do so. In order to develop those skills, one should strive to write as much code as possible before running it, and a high iteration time assists with that.
Also, I'm tempted to continue the title with "...unless it's APL".
After 13+ years of C#, on simple projects I can code for a couple hours without compiling and get the pleasant surprise that my code work the first time I run it. And I'm certainly not a genius, I'm a 1X programmer.
For me it is crucial to always have have the code in some compile-able and testable state. I iteratively work towards the simplest, most naive solution that gets the job done. After that it is refactoring time and I form it into a proper solution.
The reason I do so is because only after having coded the naive solution, I have an concrete and proven correct understanding of the task and can pick the appropriate abstractions. I always see colleagues with an less iterative style use premature-abstractions and over engineering on the basis of "we might need it" instead of knowing.
I do invest some time into thinking about the right data structure though. (Not much about algorithms, as they mostly follow from the data structures anyway.) Also if I code a well understood problem, I might use a more top down approach.
(Not a critique of your workflow, just wanted to share mine.)
Additionally for some reason on my machine, maybe 25% of the time it gets stuck while building and I need to kill it and start again. Usually I go and get coffee while it's building, but there's only so much coffee or Reddit you can drink. Many times I've taken a 30 minute break while it builds and tests, then come back to find it got stuck and need to start again. Anytime I have to work on this project I try to put it off as much as possible, as I know it's going to be painful...
With short iterative changes I have unit tests and fine grained git history, to untangle it.
Yeah that's what I'm doing all the time. Just constantly testing the thing after every little change. If the test is fast, I like it. Probably there is some dopamine thing. I have thought of it in the past as like a slot machine.
> and the code written as a result of this looks exactly like what you'd expect:
Why would you expect any particular thing?
> it barely works, and is full of redundancies, "dead ends", and other evidence that its author was probably not thinking of anything more than the next line or two when writing it.
My code isn't like that though. Maybe that code was like that because it's beginner code.
> In other words, reducing the iteration time reduces the motivation to get it right the first time
I'm always hoping it works on the first time but you know how that goes. Even when it's close there's some little thing. The hope of getting untested code right on the first try isn't really something I entertain too seriously anymore. Short iteration is lovely.
no need to spend 3 more minutes looking at the code when a 2 seconds build will tell me there's a Segmentation fault right there, or when Valgrind will tell me I missed a free ().
Came here to say that. But jokes aside, as an APLer I wouldn't be able to bear waiting for things to compile. I regularly edit functions while they're on the stack, then have them continue…
But at some point you are better thinking about what to write, how to write it, which structure you will use. And this skill has to be acquired as well.
I even see the mindset spreading to other activities, like writing an abstract, paper, proposal or preparing slides:
I regularly (more often than not) see students using their supervisors as "CI-chain", submitting iterations with lots of stupid errors, half-baked sentences, unclean structure... because they learned to hit CTRL-b and see what comes back.
Of course there are reasons to talk about half-baked ideas, but if your task is writing a proposal or a section, you should at least be happy with what you wrote before you send it to the Prof, you're wasting his time.
But I also want to say that CI, unit tests and quick iterations are still helpful, even if you know how to approach a problem (i.e. think, research, draft, test, revise, etc.)
Experienced developers typically don't sit like Hari Seldon predicting branches 50k instructions into the future with some extremely elaborate grand master plan, they're so fluent in the code and architecture that they don't need to. This is tacit knowledge that is acquired along the way, I don't think it can be meaningfully practiced.
Maybe you are saying that there is no deliberate method of practice, but I think there is: whenever you correct one of your mistakes, try to figure out how you might have written the correct code the first time. Often, it might be a boneheaded mistake, but a significant number of cases have something to learn from. Can you make an argument for the current version being correct, and the previous one not? If an API did not work the way you expected, did you overlook something in the documentation? (Maybe just that the documentation was ambiguous, and you might have realized that earlier.)
I think OP's point is that slow turnaround encourages that sort of thinking, and it does, though it is quite a cost to pay.
It's not that the knowledge can't be acquired, but this is a type of knowledge often can't be explicitly practiced through some training routine.
A beginner may of course learn to check off common errors they've been told about, and may arduously step through the code in their head line by line, but an expert sees the code is wrong unconsciously and understands what the code does without stepping through it. The only way to learn that is to engage with code for decades. There are no shortcuts.
There are no shortcuts, AFAIK (though I think most people can achieve most of their potential in less than a decade), and I am sure you cannot get there through rote learning. Are you suggesting, however, that the sort of approach I am advocating is not very helpful, or just that it cannot be explicitly practiced?
This made my day, thank you.
Or lisp. Or smalltalk. Or forth. There's an entire subfield of computer science, headed by some prestigious people, who believe that working in a REPL is better precisely for these reasons.
At this point I deploy the left-handed user argument: different developers probably find these tools and workflows have drastically different effects on their productivity.
Perhaps also a "code kata" thing. It may be worthwhile to make beginners do each for a few months just so they have the experience (and find out their own "handedness")
On the other hand, at least it's a motivator to look at the build pipeline every once in a while and see if there's anything that can be improved.
That said, working in a typed language with good IDE support definitely helps build up the confidence that things will work as intended before you hit compile.
Sure, with stick you become aware of how the car works, the engine, torque and clutch, and you get more control. This might be good for learning.
But automatic lets you pay more attention to the road ahead, whether you are a new or experienced driver.
https://www.researchgate.net/figure/A-comparison-of-accident...
From the paper:
> whether looked at by human factors leading to accidents, by driver age or by accident type and age, the results all support the conclusion that, among all accidents, the accident rate for AT vehicles is twice as high as for MT vehicles except for head-on collisions
> But automatic lets you pay more attention to the road ahead, whether you are a new or experienced driver.
Contrast to manual drivers, who will happily drive along without music. Far less prone to distractions and far less likely to touch the phone.
One could argue that the focused people who enjoy driving do prefer a manual transmission, to which I'll agree, but I'll counter that the manual transmission itself fosters that focus and enjoyment in the driving experience. The boring automatic is what invites distraction and does not foster enjoyment in the driving experience.
I know a bunch of people that prefer manual transmissions. And every single one of them also prefers loud music while they're driving.
As a more general thing: you can learn a lot by removing tools that you usually use from your toolbox. For example, going from a managed language to manual memory management. Going from an IDE to shell commands. Removing tools like language servers. Each time I did one of those things, I learned a lot. However, there are lots of things to learn and I don't know if this was the "most effective use of my time".
Another example of hard to balance learning: I'm currently working for a company. Should I focus my learning on their business and our codebase, or on general technology? General technology is a transferrable skill, and I'm scared of being "trapped" in a company if I focus on their specific parts too much. But on the other hand, companies want to recruit efficient people, and I think focusing on the business and our codebase would make me more efficient.
I'm a junior engineer, so maybe that's due to a lack of experience. If anyone has insights to share about that, I'd love to hear them.
While you might learn more if forced by the constraint, the opportunity cost is quite high, and these days you'll never replicate the experience of going off with your glass plates and camera packed on a mule to photograph the Sierras.
Not only all of that... but Ansel Adams spent a TON of time in the darkroom, and pretty much invented most of the techniques that Photoshop emulated in their early versions. Even faced with limited chances at exposing a negative, he iterated the heck out of the development process when making prints.
I share your view that there is value in the focus, but I don't think that anyone can afford the opportunity costs.
Once you're ramped up, yes, it's more efficient to produce more code and fix the odd issue that crops up. I rely more heavily on high level, slower running functional tests on code where I've been the primary author or have worked on the system for a long time. But just be aware of what you're optimizing for: tenured employees.
I wouldn't kid yourself that it's "deep reasoning", beyond people on a dopamine kick, or beginning programmers working with simple data structures. You can't reason much about a system you don't know, because everything is details - millions of lines of code, you can't mentally evaluate it. It's not until you've built a mental model of the moving parts, and are able to mentally abstract away details not relevant to your task, that you can think things through in the large.
I think just as you get more experience you tend to write more code before you run it. I've written an entire (relatively small) application and written the tests for it before I even thought about running it. Run it, few tests fail, iterate a tiny bit, done.
It's just natural as you get more experienced that you can reason about code in your head without resorting to running it to confirm it.
A high iteration time [a.k.a long feedback loop] leads to distraction more frequently than to “deep reasoning”. Human brains seem to be wired to find distraction after about eight seconds have elapsed. [1]
Furthermore, let’s not forget that an IDE doing a lot of error checking real-time, often including partial compilation and execution, so nobody running an IDE is coding for a long period without compiling, they’re more likely coding for a long period without integrating.
If we reframe deep reasoning as “stay engaged with the problem while waiting for some feedback”, we can see that there are a lot of tricks for this: Unit tests, for example, or using a different medium like pencil and paper or a whiteboard. (I’m sure this group can come up with a great list.)
[1] “Response times: The three important limits” https://www.nngroup.com/articles/response-times-3-important-...
This was a thing learning to type, for me. Our class would transcribe from paper to document. Those that watched the screen were slower, once folks learned to touch type.
That is, learning to get into deep thought is a skill that eventually doesn't want the distractions. You want to focus and produce, then review.
My two comments however would be that when you don't know your context, lots of probing is required, and that can take a lot of iterating. In industry work you're very often working on a small part of a larger software orchestration, and it's often entirely new to you when you're asked to work on it. So you don't get access or time to understand the full scope of the software to reason about. So you need to "discover" your context as it actually is at run time. This is especially true when bugsquashing.
My second comment, is that we should all have a good debugger. Having a debugger is like having a superpower. I can come into a codebase I've never encountered before, step through as it runs, gain enormously useful context, and solve the problem I wanted to solve, without ever running the code to completion or having to read every bit of logic.
If I'm working on someone else's codebase, usually I spend time reading to figure out the general architecture, then dive right in with a debugger and start stepping through. I find how to trigger the code paths related to my task, see what's happening as it happens, and get a really good understanding of how it's all running.
In what might be another controversial opinion, a working programmer without a debugger is like a working carpenter who's chosen to do everything with a knife. They may produce good work with their own way, but they could be faster with the proper tools.
But definitely having a phase where you are dealing with long build times helps you enhance one aspect of software development!
At least for small delays, say 30s-3m, I'm not tempted to go for that (other) dopamine shot of checking messages or whatever at all. I keep thinking about the problem I'm solving, what to try if the thing I'm currently trying doesn't work, and so on. I don't remember ever feeling limited with such cycles - literally only when fiddling around with CSS, which I quite often just do in the browser's dev tools for the fastest possible feedback cycles.
For me slow feedback loops push me into dopamine seeking activities, intentional breaks push me into relaxation and deep thought
Recently I got a treadmill desk and the physical exertion reminds me to take breaks where I can get off and relax and potentially think about a problem
Slow tests running causes me to not write tests and look at YouTube
There's also a "sketching" element with faster iteration. On slower iterations, you're engineering. The engineering skill is often underdeveloped but sketching is more valuable without a dedicated designer.
Also some things are poorly documented, often things like UI and browser code or legacy support for say, older Android OS. So even if you have your calculations right it ends up wrong.
My first programming experiences were BASIC on TRS-80, TI-99/4a, and similar machines. It was instant feedback, for the most part, and I enjoyed being able to make small changes and seeing the results. I was around 12 years old and that worked well with my attention span at that time.
Agree.
I think what was happening was the the lack of feedback fit the mental models of the people who were attracted to computers early.
For another example of this,Ken Thompson famously said that he never quite understood why anyone would want to see the whole file they were editing instead of the single line.
But instant feedback attracted more people to programing.
I'm one if the people that thinks that coding in paper is a great learning task. I tend to not ask people to do that in practice because it's a very effective way to lose control of a classroom, but I do use the second best task that is asking people to explain their code.
Anyway, if thinking about the code is not your focus, the increased feedback cycle is all cost and no gain.
Let me tell you, time to iterate on a huge 50 year old code base is through the roof and reaching out into interstellar space. The resulting code is pure and utter trash.
I would kill to have to have a quick feedback loop and at least be able to address quality issues through training, culture, or technical controls.
The alternative is fighting against a giant, heavy machine with a lot of inertia in the wrong direction.
This only works once you're a bit more advanced, but the one sure sign of a senior dev is the accuracy of their mental model. Which is also what allows you to write your piece without a continual back and forth with the compiler.
This all said, I greatly enjoy ping pong with a repl or instant hot reloading when doing visuals or sound.
As an example I inherited a codebase about 2 years ago that would take 15 minutes to run the test suite. It was painful. It was just long enough that it would waste an entire day just trying to implement a simple feature. The project was behind schedule and no one wanted to work on it because it just took too long to do anything.
So I spent a few weeks running benchmarks on the tests and figuring out where the slow down was. I did a bunch of different things, reusing SQL tables, only truncating tables ones that were used, refactored how the tests worked, removed some code so that docker was not needed to do the testing, etc.
I got those tests running in 3 seconds down from 15 minutes. And that changed the whole velocity of the project.
Always sharpen your ax.
I don't know what drives that, but in most jobs I've had to fight to do the right thing.
Just recently I've made some optimizations to get a functional test suite from 65 minutes down to 3.6. Parallelism is my favorite lever, as it scales so well, and core count is still going up!
Running tests is the same type of thing!
That makes each test start at the same, clean state.
2. Rolling back a transaction is not exactly free.
3. Does not allow running multiple test simultaneously.
It does with almost every form of transaction isolation mode. Read uncommitted in Postgres still behaves as read committed, so in Postgres you're never missing the mark here.
> 1. That wouldn't work if code under test uses multiple transactions.
A lot of abstractions interpret nested transactions as save points for this reason.
It's not the only valid pattern, but it is a consistently behaving one that's straightforward to implement and gets you most of the UX you want from integration test design.
It doesn't let you test interactions of parallelized code doing independent/interacting database workloads, but most web/CRUD apps don't do that.
Disk is cheap. CPU and memory are the expensive assets.
I can do about 4-5 hours a day of "difficult" programming, but then I can easily do another 4-5 hours of this maintenance type work, and the cumulative effects of repeated maintenance time keeps everything running super smoothly.
Why bother?
Even if the whole project has no career purpose, it's more fun to write new code.
So things can be largely a question of perspective.
It's also nice that bug chasing has less of an expectation of giving good times estimates, which the software industry is famously poor at anyway.
Even more so in a lot of orgs maintenance work gets hit by the "cost center" mentality and management only rewards not hearing about it.
It moves the deletion as a means towards the coherence rather than as an end in and of itself, and also includes refactoring as an option.
I hope it helps the less experienced developers grab the intention more explicitly.
And rightfully so :)
Tend to get a boost in application boot/startup time, too
The OP mentioned using tests instead of the entire game. You could also consider splitting the tests. The code base I'm on now has 9 test suites. It would be much slower if all 9 were merged into 1. So, if you have a monolithic test suite consider breaking it apart.
Our team also has a CI/CQ so I never run all the tests myself. I run the tests I think my change affects. When it passes I upload to the CQ and let it run all the tests. Then I go work on something else and check later if it all passd.
In games there is often no need to run on the target hardware for 95% of tasks so choose the hardware that's fastest. In other words, if you're making a game that can run on PC, PS5, XBox then develop on PC and only switch to PS5,XBox when you have to (platform specific features, checking perf, etc.). For VR for example I'd test via PC on a link cable or on a Vive/Index/Rift and only test on standalone Quest when I absolutely had to. So much time saved.
The same is true in other places. I work a very large project that run on many platforms (Windows, Mac, Linux, Android). Building and testing on Linux is 10x faster than Windows, 40x faster than Mac. Once I found that out I switched to Linux for my day to day work and only pull out the other devices if the stuff I'm working on is platform specific.
And while this strategy might work for the folks who actually wrote the code (and thus have tacit knowledge about it), the moment they leave and/or someone new joins the team, all that "speed advantage" is lost and it turns from minutes to hours to days instantly.
Automated tests really are the only way to capture the business logic of any code for it to not to become "legacy code" before its time.
Does that code counts as legacy?
Sure it is a bit verbose but dam is it easy to understand due to sheer bluntness and lack of magic.
"legacy code is the product of ... when your team turns over faster than your code turns over." [1]
1. https://www.software-engineering-unlocked.com/legacy-code-mi...
Actually, currently am about to be asked when the feature will be done, whereas i've decided to actually write tests for it (reusable library code across multiple projects). As far as the reality is concerned, the people who say that writing tests "uses up" time are right: being slow now will be more visible than no one understanding why development takes a whole lot of time down the line, which is when many might just shrug their shoulders and go "legacy code". What's the benefit of making others able to develop code faster, if that makes you be slower - both facts probably being visible to management, though oftentimes without the underlying reasons behind those being relevant to them?
I'd argue that that's why even governmental projects in my country (that are often contracted out to random companies) oftentimes have poor test coverage and just generally not a lot of thought is put into the quality and sustainability of those codebases - they got paid, they could iterate reasonably quickly while the contract was ongoing, why should they worry about anything? The people who take over the project then can also deliver features more slowly, not only because of the poor domain understanding, lack of ADRs, lack of documentation, but also lack of tests - and still often get paid on a time material basis. Thus, no one actually wants to improve things, apart from me wanting to tear my hair out whenever i'm expected to work on garbage like that and fix their problems.
Thus, you see some people in the industry adopt an egoistical approach to it all - join a company, focus on the speed of their own iteration without thinking about the project in the long term, spend a few years doing this and then leave for another company where they'd do the same thing. It might be a cultural thing, but working on other people's projects instead of starting my own feels like losing at this point - no READMEs, no automated CI, no Ansible, no Dockerfiles, no IDE run profiles, no common linting rules, no code static analysis, no local DBs with migrations, no local setup scripts etc. Why should i be expected to toil away without results that are visible to the business because someone else was allowed to neglect the codebase?
At this point, i use unit tests and code coverage rules defensively: to prevent someone from jumping into the project and ruining how things should work with breaking changes, which will be caught by the tests, or by introducing untested and undocumented code, which the coverage rules will catch and make CI fail. Of course, depending on the culture, you might find that other devs talk amongst themselves and before long your tests have @Ignore added to them or have been removed entirely, which is the point where it might just be easier to look for a less dysfunctional environment, despite living in a country that others outsource to and therefore quality isn't a priority.
In short: i agree with your point, but i think that there are social problems at play. You need to actually care about the code that you're developing to prevent these issues in the long term. And even then, if you care, that doesn't mean that other team members will.
From my limited experience, governmental projects tend to have the worst code quality ever. At least that's what I see here. It can easily be explained by pervasive corruption — projects go to those who agree to return the largest kickback and are written mostly to steal money from the taxpayers. With such incentives, code quality does not receive much attention.
You hit the nail on its head here, my friend.
When working with others' projects, I try to add tests where I've added/modified features, so that at least I can get a sense of it all and make sure I don't break anything. And for a completely untested codebase, it might make sense to add more integration-like tests (while still using mocks for external dependencies), such that simultaneously a bigger proportion of the codebase is covered, even if at slightly longer test times.
…I just googled for “React TDD” and the top no-money result suggests finding divs in containers and spying on dom changes. It’s like testing your cli app by hooking into tty driver’s section in /dev/mem.
And when it comes to testing what your DOM looks like, even though that does often involve spying for changes (after all, you're testing whether they occur), that's not on the actual DOM but on the virtual DOM, which makes it easy and efficient to run in a non-browser environment.
You have a “data” as a model sitting between the DOM and the controller (mainly methods and watchers). There are some gray lines, like computed properties, but otherwise the controller can run without the DOM and vice versa, you can easily force the model into a state to see how the view will look like.
A lot more testable than the old spaghetti of onClick and getElementById().addEventListener.
We have so many tools to make this easy now like docker and yet it still gets messed up.
Sure they should. I often spend all day on negative numbers of lines (I always say that the best code I write, is the code I don't write).
The complaint seems to center on CI/D services that provide inefficient build times. These increase the pain of "round-trip" implement-test-refactor.
That's quite valid, as this is how we tend to work, these days (at least, that's how I work). I often take it a step further, and iterate the design as I progress[0].
But I am one that got their start in the waning days of "big iron," when compute time was the costliest and most time-consuming part of the whole process. You'd need to schedule for computer runs, which would often happen overnight. This meant that it was very important to have your code complete, and debugged, before submitting it for compilation.
Argh. I miss it like I miss a case of food poisoning. It did teach me to do my homework, though.
[0] https://littlegreenviper.com/miscellany/evolutionary-design-...
You're missing the point of the article. It's not about valuing quantity of code. It's about valuing minimizing the time it takes to determine what you want the code to be.
If it takes you a 30 minute compile loop to add a line of code, it's also going to take a 30 minute compile loop to refactor and eliminate a line of code.
The latter is actually worse. When your iteration time is high, developers will deal with it and push through as necessary to get features implemented because they have to make the software do a certain thing. But they will absolutely not struggle through a shitting iteration cycle if the only result is cleaner refactored code.
If you want nice codebases, you need a nice iteration cycle.
Actually, I didn't. Just sayin'.
I experienced this while working for a multinational.
- A long build process and linker step that takes ages. 1-2 minutes for a small change.
- Trying the change on a real device took perhaps 5 minutes.
- A review process that can take days or even weeks sometimes.
- There was a process that merged my change set which often failed and needs to be rerun several times - often 1-2 days were spent on trying to commit something where everyone was doing the same.
- There are no unit test because some manger decided that they provide no business value.
- There were no simulation tools for software be because some manger decided that they provide no business value.
- Testing would be done overnight.
With the right tools (unittest and pc simulation) I would be effective and could test my changes prior to testing on a real device. This was for me the bottleneck and not how long it took to build the project. I was testing code that was just C on a complex embedded device that I could have tested/debugged easily on a PC in 1/10 of the time.
This was really a management problem where the management didn't understand what to do to make their employees efficient with the right tools.
This was from an old job. I work a different job now with different challenges.
1. It’s easy to have a fair amount of confidence in changes without even needing to run them. Strong static typing, good/clean abstractions, local reasoning - little to no mutability, code is mostly pure functions with side effects pushed to the edges. Being able to just write a bunch of code without needing to run it at all (and having it “just work” the first time, most of the time) is HUGE for productivity
2. Excellent automated test suite, with a nice pyramid (small number of E2E tests, solid number of integration tests, exhaustive unit tests). Also excellent monitoring and alerts, with automated canary releases and automated rollback. Basically make it so that it’s hard for mistakes to fully reach production. Being able to be a bit of a cowboy, safely, is huge for productivity too. I’ve worked on plenty of systems without this, and then I’m MUCH more careful, but when you really do have this, it makes everyone so much faster, especially once you learn to stop worrying and trust the guardrails (for simpler changes)
3. Test suite runs quickly locally, and is not flakey
4. Running the app locally is quick. Also ideally I can run just the one component I’m working on (like a front end, or service, or whatever), but have it fit into a full system running elsewhere
5. Full build/test/deploy process on CI is quick and reliable
This article emphasizes 3 and 4, and they’re certainly very important, but I think 1 and 2 are even more important. With them in place, often I don’t even need to do 3 and 4 for more trivial changes - just “I’m pretty certain this works,” and then the tests all pass in CI, and I’m very certain.
Compared to systems without 1 and 2, where I have to basically change a line, then run tests and/or the app because I’m not sure it’ll work, then do extensive manual testing to be sure because the automated test suite sucks. Muuuuuuch slower.
3 and 4 are great, and necessary for more complex changes, but 1 and 2 let you ship simple changes super quick, without even needing to run anything locally. That’s fastest of all, and can still be quite safe in the right environment.
However, a lot of changes are just small, trivial tweaks, and if 1 and 2 are strong, I don’t even have to bother with 3 or 4.
I work with a frightening number of programmers who assume that they can do this... so much so that they never bother testing to see if they were correct in the first place.
For any changes where it's not obvious that it'll work in practice, despite the tests, I also run the full app locally, and test manually. Or for any UI changes, obviously. For backend changes, for some codebases, running the full app locally is required for ANY change, period - hard to reason about code, little-to-no type safety, weak overall test coverage, etc. But if you have a simple backend change in an easy to reason about code base, that has great guardrails (great automated test coverage, great monitoring/alerts, canary deploys with auto-rollback), there really is not much reason to actually run the full app for a simple change, which is a nice productivity boost. Just make the tweak, write/modify tests if necessary, and if everything passes on CI, you're good to go.
For the 8 years before my current job, I was working in environments that weren't safe enough to do this ever, and was horrified when devs would merge without actually manually testing their changes. But my current company/environment has a codebase with strong typing and nice abstractions/local reasoning, great automated test coverage, and great monitoring/alerts and canary deploys with auto-rollback. Took me a bit, but I've embraced the "no need to run the app for simple changes" approach, and it works great in this environment - highly productive and still safe enough.
I wasn't saying you were one of those people - just saying I've worked with a lot of them.
The newer tools in web dev have crushed iteration times to a fraction of what they used to be. A combination of things like esbuild, fast refresh, yarn's offline cache, and a few other bits can get your iteration times on a site you're manually testing down to milliseconds to both build and update (literally). Decent devs have been writing tests for years; the test runners could be a bit faster but people are working on that as well.
The React app I'm working on at the moment takes well under a second to build, under 10 seconds to run the test suite (could be a sign I need more coverage..), and milliseconds to update the code in the browser in dev mode. It's nice to work with. It's not even clever or special. A default Next.js app will do that. Create-React-app 5 launched a few days ago with more of the tooling too. In Vue things like Vite are based on the new fast tooling and it works well. Vue dev work is fast. SvelteKit even goes further by using Snowpack to remove the bundling step entirely. Snowpack claims a default refresh time of 50ms.
I really hope the author's article about web dev is simply "Yeah, update your tools and you'll be fine."
FIY SvelteKit is using Vite 2 instead of Snowpack
Source: https://twitter.com/Rich_Harris/status/1367577006355976194
I also once spent a week pursuing a small-but-necessary authentication flow change across two dependencies (one of them owned by a different team) and three tests, only to be fired for talking to the other team members directly (I was a mere contractor) rather than playing 'telephone' and relaying everything through my management and their management, which would have made it take a month instead.
[0] It was actually five lines of code.
The java app I work on regularly takes 3 minutes for a single line change. Then you just need to pray that jrebel will work today and actually hot reload your change. Otherwise you have another 8 or so minutes of redploying weblogic.
Of course the ideas suggested in this can help, but when you need to run an integration test which also takes minutes to run you realize that 3 lines of code isn't bad for a days work.
One day soon I will never touch Java again.
However, the experience itself feels like a positive one - even if the code takes longer to craft, it's much easier to work with, since now i don't need to worry about sloppily written if/else chains that depend on enums or runtime type checks, but can utilize different implementations as needed. Also, writing the actual tests allowed me to be sure about how everything would actually work, all the way up to discovering that Paths.get (Java) on Linux accepts almost anything as a valid path string, whereas Windows has actual validation rules, leading to many headaches in regards to the code coverage quality gates that i set up, since the coverage differs based on which platform you run on.
But i guess that my point still stands: in certain circumstances slowing down might actually be a good thing, when you really want to know how your code will work under most circumstances and make it maintainable.
Of course, on the other hand, compilation speeds and other factors in regards to the speed of iteration definitely shouldn't be overlooked either, and having everything else apart from writing tests and actually thinking about how everything will fit together be faster is a good thing! I'm not sure what i'd do if my test suite took 10 minutes to run instead of 10 seconds as it currently does. Probably drink lots of coffee.
I guess it depends on slowing down for good reasons, vs just wasting time because of tooling or other sub optimal circumstances (e.g. what was described in the article, personally i've also seen a local API service be pretty chatty with a remote DB which was slow over a VPN, yet no one had a local DB with all of the migrations in place, and a bunch of other things like that).
Offtopic: Anyone remember how fast Pascal compiled? Now that was a really nice stack to do some stuff in, it's a shame that it never got as popular as Java, or didn't have tooling like JetBrains has, it felt like a more ancient Go (which is also pretty good as far as ergonomics go).
> Recently decided on using TDD for some library code and aiming for >90% test coverage at my dayjob.
I thought the point of TDD was that you don't write code unless a test requires it? In my mind (and the little experience I have) that translates to 100% coverage, with exceptions of really annoying side-effects that are near-impossible to test reliably.
Yep, like Java supposedly being cross platform, but having different file system implementations, because using one implementation across multiple file systems just isn't entirely feasible, for example, due to how paths are validated.
Thus, no matter what i did, i couldn't get the coverage to 100% due to semantics of how i'm supposed to validate that the low level call works and because i have branches in the test, one that checks whether the correct exception is thrown in Windows and a different exception for Linux. Even if technically the code paths are covered, telling that to a subpar code coverage plugin is hard.
So, i just settled on writing code for everything myself, but having 90% coverage be enforced on a class line level to account for the weirdness.
The lack of feedback between the writing of the yaml file and validation of the structure/type/format was frustratingly slow. I would pay good money for better tooling in the Infrastructure as Code(IaC) space.
Waiting for the resources to update, fail with cryptic error messages, then slowly rollback only to then fail the rollback. Now in an invalid state, manual resource creation was required before the rollback would succeed.
The AWS cdk has improved this significantly. As a result the sun shines just a little bit brighter.
Expected time for this work was: 1 week.
(better yet to have someone regularly scanning for these efficiency related things)
It was hell. That much so that I jumped to the first half-decent role that came my way.
Something is wrong with the way we build systems that integrate too many other systems. Perhaps if we designed them with a TDD approach all the way up things would be better...or worse. Dunno. Sometimes what we're trying to do is just way into the entropy zone.
I’m of the opinion that applying great testing practices to unity programming is very important. It’s been fun synthesizing ideas I’ve encountered into my hobby game dev career.
I’m glad there are others out there doing the good work of generating content to popularize the benefits of good testing practice :)
1) Use unit tests to move quickly through to implementing The Feature 2) When The Feature is complete, run in a local environment to confirm it works 3) When that works, move The Feature to a development environment that more closely mimics Prod 4) Deploy to Prod
Each one of those steps takes an order of magnitude longer than the previous one, so should be done an order of magnitude less often. However if you are finding inconsistencies between steps, then alter local/dev to ensure more consistent testing. (i.e. if it works on Dev, it should just work on Prod, however there are always little issues).
Running in situ could also be integration testing FWIW. Depends on what it is you’re making.
Ofc if you do the code and the unit test, all you validate is that the code does what you assumed was right. But in 2 years, when someone touches something unrelated, he'll kiss you on the mouth that your test noticed an edge case that started failing.
If it's not your unit test catching unintended changes, it's your client. You may not want that :)
That's like the Groundhogs Day of programming. Living the same season over and over again, changing things until you get it right and can move on. Sure, ai guess that's kind of programming in general, but on this level it's most of your day. No developed cheat codes?
After working at a CDN that delivered a couple hundred kb objects in milliseconds around the world I thought "code is data, so why cant code be updated in milliseconds?" I tried starting a startup that could do this, treat code as data, and therefore achieve sub-second round-trip trial-and-error loops.
Did not pan out, but to be honest we never really got to testing that thesis. I still think it could be amazing, just not sure how much of the lang/ide/build/test/deploy/validate cycle you could integrate and how much you'd have to build.
IMHO this is the main thing that made PHP successful. The "edit a file -> alt-tab -> click refresh" test loop being faster than you could click.
We eventually (1 week later) got permission to pull the release from our shared (pseudo write-only) storage area but as he was project lead he tagged a new minor version for his '2 line fix' without consulting anyone and our users were reluctant to move to a new bugfix release unless prodded with a red-hot poker.
Can't say I miss working with _some_ programmers turned managers.
The best part is when I receive a new bug report from QA, since they include the network logs I usually just need to create a new scenario, register the JSON and fix the reproduced issue.
I think this is an important observation. The time where you "wait", you can allow your mind to drift off right after being immersed in the problem. In my experience, this time has the potential to give you great insights.
I think fast feedback is great, but only if you know what you're trying to accomplish. Or trying to learn. Don't use it as a tool to throw stuff against the wall and see what sticks. Once you start to do that, spend some time away from the computer. Draw a picture, formulate some hypothesis based on your mental model. Make a map before you enter the jungle of trial-and-error.
What drains my soul is when you need to use legacy or poorly designed tools that slow you down and you can't do much about. Here are real examples of times I wasted entire mornings: tools that, if you make a mistake, leave the environment in an inconsistent state, and then you need to manually fix it (it isn't documented and changes on a case by case basis). If you miss anything or make another mistake, start from the beginning again.
Way back when I was at uni I remember watching over people's shoulders at the painfully slow iteration process. I couldn't believe not a single one thought of doing anything about it. They looked more like factory workers, endlessly turning the same handle and waiting. If you spend your time doing this, you will have no time left for creativity.
The future for fast compiled-code iteration in game development will most likely be hot code reloading, so that code changes are compiled and implanted right into the running game instead of requiring a full linker run and then getting back to the right place in the game to be tested.
I don't think anybody is ever suggesting this - but I do see people suggest the opposite: that all testing should be end-to-end "black box" testing, and unit tests are a waste of time. If you actually want to ship something that works reliably, you have to do both unit testing and end-to-end (integration) testing. I've never seen anybody sacrifice integration testing. I have seen them sacrifice unit testing.
PS. Anyone did figure out whether DDA is officially in the code or is it just a matter of network quality difference between players and the server (the game stopped to be p2p in any mode)
https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
In one of those projects the code got so big and bloated that it took around 1 minute (sometimes more) to move from one line to the next while debugging. For me it was a torture, it was ok for most.
There's a reason why old-school Naughty Dog games were so refined.
I like Android Studio's approach to running unit tests: It will run them on the host PC instead of deploying them to the target, so you can avoid the often long deployment process.
Agree, continuously re-evaluate your systems, they can always be better
Step 1 Make it less dumb
Step 2 Delete a part of the process
If you're not adding step in 10% of the time, you're not deleting enough
Step 3 Simplify or Optimize
Step 4 Go Faster
Step 5 Automate
In this blog post the guy took 3 job change to DISCOVER unit tests, it's insane :( It's not like they know but don't have time, it's that he's a "champion" just for proposing limited scope testing.
It's hard to do unit tests in a large part of the area of gamedev. How do you ensure that a picture is rendered correctly? You can take a screenshot and compare it to a stored one. But then what happens if you do artistic changes that you want to do? You have to update the stored screenshots of the testsuite. Who will review that the changes all made sense? And more importantly, there might be slight differences in the output of GPUs, depending on driver versions, model, etc.
So let's say you have a bug in your game where if you walk through a level in a specific direction, the game crashes. The error is easy to check for: just make sure that there is no crash. But how do you create a reproduction of the bug? You could record controller inputs and play them back. Then a different department changes something how quickly players move and increase their walking speed by 10%. Suddenly your player walks into a wall and the test is basically broken.
Compare this to a CRUD app where you have well defined operations and their impact is well described.
That being said, even in gamedev there are areas that are well testable. You can do a unit test of the low level networking layer by trying to make a server and a client, dropping some packets, then looking if the packets still arrived because the networking layer sent them again.
You want faster iteration times in order to provide feedback, but if they're too fast then you neglect other things or end up swinging wildly. Consider a thermostat that turned on the heat or cooling at just .1 degree below or above its target, or turned it off as soon as it hit the reverse. Or if because the feedback comes in as a torrent of data you attend to that feedback instead of other elements of the system or your capabilities.
As a system gets more complicated you have more factors to consider. In the case of programming, if you rely too much on that near-instant feedback, how much are you internalizing about your system, language, and environment? How often are you making the same errors but recovering quickly (so it's not slowing you down too much, or doesn't appear to be slowing you down too much) because of the feedback? How much faster could you ultimately be if your work were smoother and not so rough and jagged?
Introducing a delay, here, gives you time to contemplate. Even if it's just taking an hour or two a day to sit back from the keyboard and ponder what you've done that day and what you will do the next. Create a plan instead of jump into action, even if the plan doesn't get executed perfectly or ends up being the wrong plan that bit of contemplation is when you learn.
But too long a delay (especially a forced delay) causes other problems. The actually needed feedback (not just compiler errors and such, but your V&V issues discovered from testing and evaluation) getting delayed by a day or more can be too much (especially when working with a team, where other parts are potentially changing around your own changes). Too long a delay also promotes batching many unrelated changes together because you don't want to sit through the whole process again. Consider a system that takes a week or a month to get feedback from an external test team, you'd be tempted to throw many changes at them because of that week or month long delay (or more!) and not just one change.
So strike a balance, find a point where your iteration cycle permits you time to think and not just act so you can really learn (both programming and the particular system you're trying to develop). Smooth out your development so that you're slowed down not by having to take corrective steps but having to ponder logical steps. "Is this the right data structure? Well, if I isolate it in my domain model then I can swap it out later and no one will be impacted." or "I'll take a walk around the building and think about what I actually need here."
E = m c^2
Any 5-year old can write this.