I’ve become worse at programming | Hacker News Reader

I’ve become worse at programming

fascinatedbox.blogspot.com

352 pointskumaranvpl115 comments

If you're an explorer and you live in a small valley, then you will slowly begin to believe yourself a better explorer as time goes on. But one day you will get to the edge of the valley and look out onto a vast landscape and believe that you have become a worse explorer.

Checkout the volume of N-balls [1]. As the number of dimensions in a "sphere" become higher the "volume" of the sphere becomes smaller when you compare it to a hypercube of the same dimensional count. This is another way to saying that as the number of dimensions increases the closer the "halfway" point of the volume of the N-ball gets to the edge of the N-ball.

Programming (and software engineering by extension) involves a lot of orthogonal issues. We can imagine this so be a really high dimensional space. My theory is that whenever you're in situation that can be represented by a high dimensional space you will learn a bit of knowledge and think you know something. Then you'll learn a bit more and discover there's more to learn. The more you learn the farther the horizon moves away. It will feel like you're getting worse. Just like finding out you live in a valley.

What's really happening though is that your increase in skill is allowing you to realize how small your world was all along. Keep learning and trying and you'll get better. It will however, take a while.

[1] - https://en.wikipedia.org/wiki/Volume_of_an_n-ball#Low_dimens...

Wise words!

This certainly applies to programming but I've felt it too in my Aerospace career. I used to feel like the smartest guy, I always knew how to do things or learnt them really fast, and I couldn't imagine where the horizon was. Now, the more I see, the further I feel the horizon is, and it's pretty frustrating as it makes me feel less intelligent.

Your comparison helped me to see things a bit clearer and to be a bit more motivated.

Isn't this effectively the same as the Dunning Kruger effect (as it applies to this situation)?

https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

More or less. I'm suggesting a mechanism for why something like the Dunning Kruger effect might occur and where we are likely to see it occurring.

That's a very interesting way of explaining it from a perspective I hadn't considered.

In the paper, ability is positively correlated with perceived ability. So we would need some other effect to explain people saying they got worse over time, or people's perceived ability decreasing over a time period in which their ability improved.

>We can imagine this so be a really high dimensional space. My theory is that whenever you're in situation that can be represented by a high dimensional space you will learn a bit of knowledge and think you know something. Then you'll learn a bit more and discover there's more to learn. The more you learn the farther the horizon moves away. It will feel like you're getting worse.

That is all true but I don't think the n-dimensions analogy applies to the particular essay the author wrote.

He didn't look back on his accumulation of learning and suddenly notice that it was smaller part of a larger knowledge space. (E.g., programmer stumbles across "monads" and suddenly he realizes that's just the tip of the iceberg in a never-ending horizon of category theory.)

Instead, his essay is lamenting the all-too-common "joy vs drudgery" in projects. It's a similar sentiment that aspiring writers who don't finish their "great novel" have. He then grades himself on his lack of motivation (words like "lazy") to stay on top of programming tasks that are not pure intellectual fun. On his idiosyncratic standard of "programming competence", he wonders if he is a "worse programmer".

Of course, he can choose whatever "success metrics" he wishes to apply to his personal projects but some of us might think that staying on top of documentation (that nobody will read) or adding features to Lily (that a large population of programmers haven't adopted) can bring undue self-criticism.

If we are to plow ahead into boring unpleasant tasks, there usually has to be a compelling motivation -- e.g. we think the personal project is the basis of a great business startup... or the code is part of a PhD paper due next Friday, etc.

Maybe there's a simpler explanation: FOSS projects can easily turn into unpaid work, and devs tend to burn out.

Projects that start as fun programming exercises eventually take on all the aspects of a full-time job with "customer" support, project planning, maintenance, bug fixing, and so on.

It can become very draining, and if the project isn't unusually popular or high profile and/or there's no other dev support and/or the team isn't solid and mutually supportive it can be understandably difficult to stay motivated.

One of the insidious things about FOSS is the way that devs can make themselves feel as if they ought to be doing all of these things, sometimes in addition to a full-time job.

The problem isn't that they're not up to the task, it's that the expectation isn't very realistic.

Then don't do it. If no one is paying you, you owe no one anything. You have no obligation. Walk away. Our society and economy have lots and lots of money. If the way system works can't justify paying you for your work, then you should not be doing that work. At least that's the way I see it. At this point FOSS is big business, if you are not getting paid, then you should not be bothering with it.

Get someone with money to pay you for it or don't do it. The end.

It's not always that way though. We often suffer as if we have unique problems, but through house repairs (got a nice long chat with a couple GCs) and extracurriculars, I find that a lot of my problems look a good bit like other people's problems. I think the surface area thing is overblown.

There are always stories about innovators who simply apply established solutions from problem space A to problem space B and everyone is amazed. The surface area problem certainly exists for the entire domain of human knowledge, but domains pretty much exist to wrangle problems that are manageable in the first place, even if it makes you myopic in the process.

I get frustrated in interviews because everyone is just so convinced that the latest utility module is so new and unique and it's really just a retread of five other things. It might have some interesting semantics that change how you use it, but even that isn't always the case.

Plus ça change, plus c'est la même chose.

I've just started telling people that 'how' ages faster than 'why'.

I think I get part of the N-ball analogy, but I'm trying to piece together the rest...

So it seems like one problem/feature with high dimensional spaces is that you have more regions which are orthogonal to one other. So, it's possible to have quite a bit of motion, which to a human mind might give the impression that one is doing quite a lot—but then you might discover a new way of turning around and find that space extends in another direction, too, at which point you realize you covered a smaller fraction of the available space than you previously believed.

What I don't get is how the volume of an N-ball with fixed radius going to zero as N goes to infinity plays into the analogy. When you say, "when you compare it to a hypercube of the same dimensional count," I don't get what in the programming analogy would link to the N-ball rather than the high dimensional hypercube. I also get the algebraic justification for this property of N-balls, but I still have no real geometric intuition for why it would be the case.

Could you clarify what's meant by, "as the number of dimensions increases the closer the 'halfway' point of the volume of the N-ball gets to the edge of the N-ball"? Not sure what's meant by 'halfway' point here, or edge of the N-ball. My guess is that you're talking about distribution of volume, like if you have a tapered cup that gets small at the bottom, most of the volume will be at the top.

Checkout the generic formula for any number of dimensions [1]. The function in the denominator is a factorial (except extended to work on decimals instead of just integers). The functions in the numerator are exponents.

Factorials grow much faster than exponents, so with an N-Ball we expect the "volume" to be small if we were to compare it to the corresponding hypercube when the dimension and when the diameter / length are all equal. After all we're comparing approximately diameter * R ^ D / D! and length * R ^ D. So if you put an N-Ball into a hypercube with length = diameter, what you'll discover is that the N-Ball is tiny when compared to the hypercube. Which means we have the weird and counter intuitive conclusion that all the "volume" of a hypercube is in the corners. Try calculating the volumes yourself using a high dimension like 100. Even better plug the whole thing into a script so you can generate a table and see the effect get progressively worse as you increase the dimension count.

There is also a similar effect on the N-Ball itself. Imagine drawing a smaller circle inside of a circle where their centers are the same such that the inner circle contains exactly half of the area of the outer circle. Now imagine doing this in a sphere. Now a 4D sphere. Now a 5D sphere. What you'll find is that as you increase the dimension count, the radius of the inner circle will get longer and longer. Which means that half of the "volume" of an N-Ball slowly moves closer and closer to the outer edge of the N-Ball as you increase dimensions.

Here's the punch line. Imagine that the N-Ball represents all there is to know about a topic. You begin in the center of the N-Ball and in order to learn more you have to walk to the outer edge of the N-Ball. Also imagine that you're only able to see what's right in front of your face. In the beginning at the center there is very little volume to look at. It will appear to be easy to understand what's going on. But as you walk to the edge of the N-Ball, the amount of volume that you're able to see increases. When you get right to the edge of a very high dimension N-Ball you will notice that you only know half of what is available to know.

Simple topics are N-Balls with low dimensions. It's easy to "walk" your way to knowing half of everything there is to know. Complex topics are N-Balls with high dimensions. Programming (and by extension software engineering) is really high dimensional. The more you learn the more you notice there is to learn. When you walk to the edge of what seems possible you find yourself swimming in a sea that dwarfs you. But in the beginning it looks like there isn't really all that much going on.

[1] - https://en.wikipedia.org/wiki/Volume_of_an_n-ball#The_volume

Cool—that makes sense! Thanks.

I've been thinking a little more about how the space of software engineering knowledge might be structured though, and one thing I'm betting is that it wouldn't be an N-ball, but instead some complex sponge looking thing (because the extent of software engineering knowledge isn't equidistant in all directions/aspects). Ordinarily I'd think the N-ball would be fine to use as an idealization, but since the hypercube behaves so differently, I bet the idealization would end up changing the character of the situation.

I'm also trying to consider what the analogy of a plane would be in software engineering knowledge. Let's say it's on two dimensions: X and Y; and let's say X is knowledge relating to text editor usage, and Y is knowledge about unit testing. It seems like the XY plane is mostly full of empty space, since most facts (single points) from either field are unrelated to the other. So, if we have a pure text editor fact X=5,Y=0 —what legitimate fact might exist at X=5,Y=1? It would have to be some single, constant piece of text editor knowledge which meaningfully combines with a unit testing fact. There may be some points in this plane on the line X=5 which are valid, but since all unit testing facts don't have a meaningful connection to text editing, there will be gaps.

Additionally, it seems like the way you'd navigate the space would be different than walking a single contiguous path. Some of our knowledge is map-like and can help give at least sketchy knowledge of the boundaries of the field without walking to them step by step. And some facts are dependent on knowing other first so that you have 'pass through' in a particular order, but a lot of the facts about a single subject are still unordered, so you would be doing a lot of teleporting.

Just for amusement, here's another idea for an alternate way of spatially structuring knowledge: each general principle is some high dimensional object of its own; each axis corresponds to a parameter to the general principle. So you have a general structures like: ProgrammingLanguage(syntax, semantics, runtime, standard_library) —each of the parameters is a different axis, and you always get a meaningful ProgrammingLanguage back if you change to a different position on some axis (seems like this structure is recursive though since each of the parameters could be a general principle itself, so I guess these objects are nested?).

When you learn things you are jumping around between these non-intersecting objects (which are even in differently structured spaces) without realizing, until you inductively discover these general principles/categories. Additionally, the speed with which you move around inside an object increases dramatically once you acquire knowledge of its general structure (since learning new facts which are just different instantiations of already known general facts is easier for us).

I wonder if there is some kind of 'meta-space' that would be useful for describing the relationships between all these objects which exist in spaces of different dimensions. Also not sure about the geometric significance of the 'general principle objects' being recursive...

Edit: seems like the recursive/nested 'sub-objects' would just be lower dimensional slices of the whole higher dimensional 'general principle object'...

Yeah, I like the way you're going. I haven't spent much time thinking about how to fit in software engineering, but I did try to think about how to structure programming issues (which is probably the first step if you want to eventually get to a software engineering).

https://www.sep.com/sep-blog/2017/04/25/objective-code-quali...

It's missing a conclusion / wrapping it all up (which hopefully I can get to soonish), but is otherwise mostly done. [I should probably augment it with motivations and examples.] I'm not sure that this is the definitive answer, but more like this is my attempt to structure some of the things I've been encountering.

EDIT: Sorry about the length. I'm not sure there's a good way to shorten these sorts of ideas.

Interesting! I'm on "Problem Analysis: High Dimensional Spaces" so far. I'll probably just leave a comment there if I end up with something to say since we'll probably drift too far off topic if continuing here ;)

Edit: I finished but couldn't find a way of leaving a comment on your blog. I thought it was an interesting read—but yep, definitely needs that conclusion! Hard to say much without that. Is your goal to use the ideas to create a new metric for how difficult a code base is to work with?

To use your analogy, when I start exploring outside the valley, I think to myself - "wow, a year ago I only knew this valley. Now I'm getting to know this new area! I've come a long way and I feel great I've made progress!".

I tend to focus on the fact I am constantly expanding my knowledge, instead of worrying about all the stuff I don't know yet.

I still consider myself a generalist, however. I can't say I'm an expert in any one area. Maybe that's why I feel the way I do.

great way of thinking. its all about the journey

TL;DR "The more you learn, the less you know"

It is more like traversing a tree. The more you learn (the deeper you go) the more branches you can explore which leads to the conclusion that the more you know the more you realize how much you still don't know.

We live in a time were everything gets "community reviewed", jr devs like to post everything they do on github/bitbucket/etc, fork someone else code and comment on issues aka "Social Coding" just to get people to "praise them" and to score "internet points" but historically coding has been an a "alone skill", despite years of people advocating "pair programming" or some "agile" shit, the most useful codebases come from devs that isolate themselves on a topic, solve the problem and came with a solution. In the old days i used to code "things for me" not for everyone but since the era of "Social Coding", everyone pressure you to code for everyone and i have been only disappointed with people bashing "my code" because it's not "unit tested", made in "go" or "rust", etc... my advice for everyone that feels the same, is to stop "contributing" on github and start doing things for themselves, stop listening to the crowd and start living outside "the grid".

I have encountered this too. Some of most useful personal coding time is spent to exploring something new or solving a real problem with a quick and dirty solution. At no point do I set out to create a full polished, deliverable thing. I don't even know if I will solve my problem or understand the new thing! Why publish such code? A carpenter doesn't varnish and sell every single bit of wood that is practiced or worked on nor an artist present every single sketch.

Now imagine that to be hired as a carpenter you had to first prove that you have worked on dozens of pieces of wood before anyone would even consider you. Of course, the analogy falls apart here because we can expect most carpenters to have produced finished products, but most finished software products are highly complex and can take years to complete. Worst of all, these products exist in a marketplace where cynicism reigns: "Why did you build that? We already have three hundred of those. Quit reinventing the wheel!"

Do we complain when a carpenter builds a nice stool because there are already enough stools in the world?

Many companies now expect a GitHub history, as if that alone determines your value. It's no wonder people publish everything. They need to establish that they're doing something, even if it isn't master level "carpentry."

> Worst of all, these products exist in a marketplace where cynicism reigns

For better or worse, that cynicism is a natural outcome of the environment.

Imagine if there was a global service in the furniture market, called OS, to which you subscribe, and it lets you browse through a catalog of all pieces of furniture ever created, and order any one you like, for free. It's still desirable for carpenters to hone their crafts by making n-thousandth iteration of a stool, but were one to publish such a stool to the OS, everyone would immediately ask: "why did you publish that? We already have three hundred variants of those!".

This is how it works in computing industry. People will commend you for doing side projects for learning, but if you try to release something as a product, it will be immediately evaluated against existing alternatives. As long as you mind the difference between publishing code on your personal site (or Github profile, etc.) and publishing something as a product for others to use, you won't encounter the criticism.

I don't know how others approach Github profiles, but personally, when I view someone's profile, I assume everything there is side projects and playing around, unless it's clearly marked otherwise.

> Worst of all, these products exist in a marketplace where cynicism reigns: "Why did you build that? We already have three hundred of those. Quit reinventing the wheel!"

I've been sitting on a bit of rhetoric in case that comes up, though I've been blessed not to need it. Regardless, let me share the rhetoric -- just in case someone finds use for it:

"Put bicycle wheels on a cement truck. Put airplane wheels on a skateboard. Put train wheels on a wheelbarrow. The metaphor about reinventing the wheel doesn't go very far -- because the wheel has been reinvented many times, and thank god for that. I like having trains and planes and bicycles and trucks and skateboards and wheelbarrows. The world would be poorer if they all used the same wheel."

Worse, people frequently put truck wheels on wheelbarrows because they're the new hot thing.

Edit: Thanks for sharing. I will use this some day.

Yes, I think this very thing when people try to commoditize software components by analogizing them to a wheel. It works in some places (I wouldn't want to write a custom Redis or an RDBMS) but in most places, it falls down.

All the people I've heard use that analogy don't know what they're talking about when it comes to software development. I think it comes from the "Taco Bell" MBA types. They're the same people who think WordPress will put all web developers out of business.

It's frustrating to see the art of software development regress so much.

> I think it comes from the "Taco Bell" MBA types. They're the same people who think WordPress will put all web developers out of business.

And then on the other end of the spectrum are 'programmer types' who think WordPress is never a good solution because it's shitty for their particular use case.

Unfortunately most businesses these days rely on MBAs to make those decisions. Most MBA tracks don't even offer software development courses. They offer Accounting, Finance and Marketing, but not software development. That's funny because all of Accounting, Finance, Marketing and especially Operations run and rely on software systems, many designed and/or customized in house. It's a huge gaping hole in management knowledge in the US and it shows.

http://www.hbs.edu/coursecatalog/

Start-up managers don't want a "master carpenter."

They want a replaceable journeyman who smacks a few nails and strings together a few prefabricated solutions.

There are pros and cons. And, as for stools, well, these journeymen "coders" have polluted GitHub with quite a lot of stool.

thank you for yanking the words out of my mouth, thank you for restoring some of my faith in humanity, this comment just fucking rocks!

I've been extremely depressed about this whole 'thing' you described and I wasn't really able to pin it on a paragraph or thought. When I read your comment I literally saved it and said to myself "there are people out there that think like me, i am not alone."

Thank you. I don't know if I'm not sounding rational but I just really wanted to acknowledge your comment. keep at it.

https://www.youtube.com/watch?v=Go-jJlGd1so

Different problem spaces call for different solutions. Many of the problems programmers at medium-large companies face in practise are problems of communication, coordination, consistency, and code organisation.

So the question to ask is whether your code is having the impact like that which it should. Does this code make you (and/or your employer) enough money? Does it have enough impact on the world, if you are not doing it for money? If so, then you are righteous in this course of action. Otherwise, there may be a risk that you're That Guy who thinks he's all that but isn't and ends up on thedailywtf.com. We really can't tell from here :)

Most of the things you complain about are tools for helping a team (or a community) work together. If you do not do that, and instead produce software in isolation, you may in fact have no need for practices like code review. You may also have a reduced need for unit tests. It depends on your product's lifecycle and complexity.

Myself, I'm working at a payments service provider moving <BIG> amount of other peoples' money daily. We have unit tests. We have code reviews. We try not to break our stuff.

(That said, choice-of-language whining on a project that's not yours is almost always unalterably lame.)

You are talking about programming at work, this is a different place that requires a different mindset to approaching problems, m talking about side projects, the ones you do to learn, fun or to solve a common problem you have, developing at work it's usually the only place were you can't innovate, nor "become better at" something, it's driven by experience and profits.

PS: Without offense but you sound like a management robot, all those "big words" usually come from project leads who don't code, just "integrate" code.

"big words"? are words like "lifecycle" and really project-lead words or are they just indicators of "i have worked on soft-appliance software that releases on CD periodically, and also with continuous-deployment web services, and understand this as a defining factor?" Is that really any more ridiculous and managementy than putting "code review" in scarequotes? :/

And your point is? besides self-aggrandizing yourself there is no point related with the original topic of my comment nor yours.

This is exactly the way I've gone - I have a "projects" folder full of shit that just solves a problem for me in the quickest and mentally easiest way possible. I go learn new technologies only when they improve my ability to do that.

I've started doing this as well, and I'm finding often my solutions end up being simpler, better and have less dependencies than the go-to solution anyway.

For example, here's a React 'collapse' component, as people commonly use for accordions.

https://gist.github.com/JonathonAshworth/b401810b965149348d0...

It has 1/10th the code of rc-collapse (the 'go-to' solution it seems), and 4 less dependencies. And it's simple enough that basically any competent Javascript programmer can edit the code if they want slightly different functionality.

Importing NPM modules for this stuff is madness

Amen. "Everybody can code" is toxic thinking. If you can't chisel down a problem without importing half of NPM and a sprawling tour of GitHub/StackOverflow, then you are part of the problem.

Let me see if I understand the author's definition... in this essay he's equating "better at programming" to mean being better at management topics such as:

- SDL (software development lifecyle)

- technical debt

- documentation

- shipping mindset (e.g. "real artists ship")

However, if his Lily programming language is a personal pet project, are all those management disciplines applicable?

For many of us, pet projects are there to explore new knowledge of programming ideas. It's an intellectual exercise and not a management one. It's the projects at work that require discipline of meeting deadlines and reducing the backlog of defect reports.

15 years ago, I started a "music CD database" project to catalog all of my discs. I fiddled around with creating some database tables and some prototype GUI screens. A bunch of other unrelated things in life got in the way of working on it. These days, there's Spotify & iTunes which means my pet CD project will always be "unfinished" and neglected. As a result, I can write 2 different essays: 1) My abandoned CD project proves I am a worse programmer, or 2) My CD project was interesting to me at that stage in my life but since I've moved on

1) A lot of the most valuable things I learn on my personal projects are how to be better at those programming management topics (and don't forget automated testing and reproducible research), because becoming better at managing my own projects creates a multiplier effect on everything I do.

2) By becoming a better manager, you reduce the friction involved in achieving your goals, and with less resistance to action, you are less likely to have your life interfere with getting things done.

3) For example, unit tests and documentation let you come back to a project after a long break and easily pick up again where you left off. Without them, it might require too much effort to get back into it, and poof no more project.

I wouldn't call these "management" topics. I would call these the difference between a professional developer and a mere coder.

The author is clearly feeling burned out and has lost motivation for his project. Working on something for 6 years definitely shows persistence, and persistence is never blind, there is often something fundamental that is driving people working towards competition.

Whenever I feel like this for a project, which happens sooner or later when you don't have users. You should take a break and redefine priorities. Excitement fades when you are prioritizing objectives that don't bring any real concern-able outcome.

Take a step back, see what is at the heart of your project, and see if it still excites you, is it something worth the struggle and effort? If yes, find a better way to do it or complete it with the way you already know. If not, move onto something new, else you'll keep getting more frustrated because of slow progress and lack of motivation.

I'm not sure whether the author wants to find the motivation to continue or the will to stop.

This article is frustration embodied, and I completely empathize, having been in a very similar situation with a large and long running project that had many (thousands of) users. Motivation and enjoyment comes and goes, but I couldn't bring myself to stop doing it because of the effort it took to get there, and because people were using it and counting on it's continued development.

The author is sad and tired and can't see the big picture. It's extremely unlikely he's getting worse at programming, unless he works really hard not to learn from this -- but this article is evidence he's already learning what happens to big projects. You have to acknowledge what's going on (done) and then step back and think about how to improve the situation.

I'm not sure how to find the motivation to continue, if that's what he really wants. But at least take a solid break and then spend some non-dev time re-evaluating what's important and where you're headed. Consider whether continuing solo is a goal, or whether recruiting real partners might be what both you and the project really need. Ask what success looks like.

As for permission to quit, it's not true that giving up is a massive loss of work. All the work done is already done. Lily runs, and lessons have been learned. The code isn't going anywhere. You can walk away from this and you get to keep everything you ever put in. What you lose by quitting is future potential, not the sunk cost. So if you want to quit and the only thing holding you back is fear or sunk cost -- go ahead and quit, sunk cost is imaginary in this case.

> motivation to continue or the will to stop.

Either one. I keep imagining quitting, but then pulling myself back. That thought of it'll get better in a month, or two months, etc. I'm hoping to motivate myself to march through as I once did, or to drop it and not look back. Not to be constantly conflicted.

That's interesting, I didn't quite expect that answer! Obviously I don't know your exact situation, and I can't hope to offer actual advice, but I can still empathize, and pretend to help, and suggest mental frames that have helped me. Imagine that everything I say here is me talking to myself rather than telling you what to do. :) Like a lot of people, I'm way better at seeing how to solve other people's problems than taking action to solve my own.

There are a lot of lessons you can take from this project and leverage for later, on this project or other projects or for your current or future day job. If you leverage your work here for something else, and quit this project, maybe knowing your experience helped you transition will ease your mind about quitting. I'm serious about sunk cost being imaginary; you aren't losing anything you have by stopping development. The outcome was always going to be what you learned by doing it, and you have that.

Not sure, but it occurred to me that one thing you may be discovering is that interpreters (like compilers & OSes) are a different beast than apps programming. You can't necessarily use the same coding styles or keep the same clean abstractions. You are bumping into project scaling problems. We all have a point where we start producing spaghetti. Everyone. Some people have some tricks for going further, but nobody can do it forever. Ever read this post? https://www.teamten.com/lawrence/writings/norris-numbers.htm...

What you're learning about the discrepancy between what you expected to happen 6 years ago and what has actually happened since could be redirected towards a management philosophy, or even a job doing that, if you were so inclined. You mentioned you are antisocial (and we all are to varying degrees) but what's happened to this project is what happens to most projects. Most software gets ugly and less fun over time. In fact, it's probably all software.

I freaking love coding, it's addicting. But software as a whole has been making me mildly unhappy in part because it's so difficult to feel like I ever finish anything, whether I'm in a large team or a small one, or working solo. Many days I feel like I could code every waking second I have and not end up where I want to go. I have big plans and, more and more clearly every day, not enough time.

BTW, I haven't gotten rid of my constant conflict. It might not go away unless you decide to end the project. Mine is still there, and I'm still going. But I have slowed my project down and I spend less time on it. It's helping relieve some of the pressure, and helping me be more judicious about what I do. I'm starting to spend more time doing side things that are fun & interesting to me.

Maybe do a trial quit, and change your mind if you want. Walk away for three weeks and don't think about it for one second. See if you're drawn back or start to feel free.

Is the author is suffering from depression?

Don't get me wrong-- most of his sections would sound familiar to any maintainer having to deal with growing technical debt. But the section headings "Communication" and "Resigned" are certainly red flags. And I've furrowed by brow four times now trying to grasp the connection between the author's astute observations about code maintenance and assertion, "I've become worse, not better, at programming."

Anyone know the author? If so it's worth sending a message to check in.

Yeah, it's depression.

There are people in the mental health profession who can help you with that.

I don't really want to pry any further than that, nor encourage others on this list to attempt to problem-solve your situation.

I do want to say it's disappointing to see no other comments picking up on this.

It's a mixture of problems, this just being one. That's as much as I'm comfortable talking about here.

Your thoughts are much appreciated.

self diagnosis or by a professional?

If the point of personal projects is to push yourself and learn, you've done that in spades, you should feel proud of your effort. I think it's normal for a project like lily to wind down after a time. Particularly if there's little or no uptake or contribution from others. I (and many others I wager) have a much poorer language implementation in my archives, only difference is I never made mine public, as you did on github. Kudos for that as well, putting it out there for public scrutiny.

Re closure implementation, I suggest taking a look a how they're implemented in the Lua source code. Also, the documentation for Guile has a lot of really nice implementation detail. They're not trivial to implement, kudo's on doing your own thing, but this is an instance where I think studying the work of others can pay huge dividends.

In my 30+ years of experience with software development, computerisation, and general technological prowess, applied, I have this to say: yes. But: no big deal.

These things happen in cycles. Computers are nearly, entirely, a social substance. There is nothing at all valuable about computers unless there is a real social component.

Because of this significant aspect of its makeup, computing is necessarily cyclic. The shadows of industry, as well as many many bright lights of academia, shine directly through the world of computing, and specifically, computerisation.

You see, computers are absolutely useless, without the user.

And users change. Societies change. Markets change, people change, attitudes change, and so on. This is a natural state.

Users change.

Its the User who has become more demanding of the programmer, while individual developers have the danger of staying, relatively, static and attached to the things they know. The things they invest in.

Alas, its easier for a User to switch from one stack to another, than it is for Developers. Usually.

So .. all I really want to point out is, this is normal. Embrace the change of civilisation that is happening around you -all computer technologies, productive+ or -productive, are a consequence of the progress of humanity, as a whole.

"Fires everywhere" is an excuse that you can hear in 99% of offices around this planet. And it's simply not true. 99% of people don't have "more fires than most people" by definition. That's how it becomes 99 out of hundred.

The only thing one needs to change is this: Don't cover the fire, extinguish it. That means the first fire you handle you stick to. Don't let a second fire take away your attention from the first. Then find the source (e.g. by asking yourself and colleagues "five times why"). Then find a reliable, resilient solution for that cause. Then repeat until the fire is extinguished. Only then take on a second fire.

If you start doing that, well, of course the first few fires will take you a lot more time than any fire took before. But that's the way to reduce the amount of fires.

And the great thing about it: If you work like that it doesn't matter if you extinguish a fire in two hours or in eight hours. Since extinguishing a fire has an iterative payout over the next few dozen years, a small delay of even a few months doesn't harm you in the long run.

The only exception to this rule is paying your bills. If you really, really need to fake a fireless environment in the next two hours to continue surviving (e.g. as a company), then go ahead and cover all of them. But that happens very rarely. Most fires only seem urgent because urgency lets us feel important. And immediately after the urgency you can continue with the actual process of extinguishing fires.

First time there is a fire, put it out. Second time put it out and do a protective measure. Third time increase protective measures, and look at addressing underlying issues (five why's or similar can be helpful). Do this (and don't rewrite everything all the time), and fires will be rare.

Not if the source of the fires is ever-changing scope or features by clients and/or BAs/Executives, especially with them trying to bypass the existing process and get the changes out the door that day. When pretty much everything is determined to be an escalation...

I've been in this situation many times.

I can agree with that addition. Not everything that seems like a problem is an important, recurring one.

Also kudos for saying it just as clearly but in 2 sentences!

Here's the reddit thread mentioned:

What makes someone a bad programmer?

https://www.reddit.com/r/cscareerquestions/comments/6crxwb/w...

The top comment on that post is incredibly disappointing.

Unpopular though it may be, things like working memory, granular abstraction comprehension, and logical reasoning capability are highly correlated with g. And g is largely immutable under "training".

Soft skills are, therefore, far more teachable than hard skills.

Maybe. I suppose it depends on the goal.

From my own experience, generally the barrier for being productive at a job was soft skills, in that if a team member was suffering from an insufficient hard skill but had good people skills/personality traits I could teach them enough of the hard stuff they are missing so they can be productive, but the inverse has rarely proved true.

Now, you can work around a team member's deficiencies in soft skills, but the difference is they aren't likely to pick them up.

I would go with "not following the advice of how to be a programmer ( https://github.com/braydie/HowToBeAProgrammer/blob/master/RE... ) for your given skill level.

There are soft skills and hard skills that are necessary as one progresses. Things like "know when to go home" are incredibly important for programmers who would otherwise spend days in the office (cot under the desk, showers the coworkers are lucky) given the chance.

"my mainstay project is Lily, a programming language, and I've been working on it for about 6 years now"

First, 6 years shows great dedication. Kudos for that!

What kind of usage does Lily have? The biggest forcing function, IMO, is true usage. Let your users tell you how good/bad of a coder you are.

Working on something for 6 years with no users is a problem. I spent about 18 months working on some web technology and never got any users. After 18 months, I realized the project was going nowhere and then I went and got a job. (Plenty of users, plenty of collaborators, and a regular paycheck.)

So, IMO, don't ask yourself if you're a good coder. Ask yourself what kind of niche Lily serves, and what you can do to grow Lily's usage base. Ask yourself what is Lily's functionality.

I didn't see any examples of real-world programs written in Lily. The problem with languages and frameworks is that the abstractions need to be tested in real-world programs to verify the design assumptions.

> Working on something for 6 years with no users is a problem

Agreed. If one of the goals is to actually have users (or fellow contributing developers), that would be a major contributor to burn out (lots of effort, few rewards, for a long time).

> Working on something for 6 years with no users is a problem

You seem to think that the only purpose of writing programs is for people to use them. Sometimes the act of constructing something is more interesting that the result, and would only be interesting to one person.

> Sometimes the act of constructing something is more interesting that the result, and would only be interesting to one person.

It's not learning if it's 6 years. You can get a university degree in that time, which is a far more efficient way to spend the time learning. It's not research either, since research is directed.

A university degree is only one narrow aspect of learning.

Please, tell me more about how much more you learn about Operating Systems by doing a three year course on them, as opposed to implementing them off your own back using only research papers to guide you

What we need is to ask the questions again. E.g. why do I need work on a new language? If the answer this time is different, even slightly different, that is because after a certain amount of effort, we accumulated experience and gained additional knowledge, and we have become better at what we do (relative to the question, e.g. programming).

We are inherently very arrogant, often assume a God-like attitude, that once we had an answer, that is the answer. That is a trap that we do not really benefit from. What we needed is a scientific attitude, that we don't have the ultimate answer (to anything), so we need investigate the same questions again and again, while reminding ourselves, that every time we reach a slightly different answer, we have become better selves.

> The worst part is that I used to have some measure of pride in what I did. Now, well, something breaks and I fix it. Done. I don't bother blogging about much of what I do since I don't find it exciting, and it's not new.

This is a critical factor. If you don't care, your project will suffer no matter what kind of project it is.

What is a good way to stay motivated? And at what point should you consider calling it quits? (Since as he admits: persisting is now making him worse at what he's doing than he used to be).

I think these are some deep and fundamentally human questions way beyond just programming.

Six years is a damn long time to work on a solo project. There comes a time when you have to just put it down and move on to something else.

This applies to any discipline. I spent four and a half years working on a solo graphic novel. Five and a half years if I count the time between finishing all the art and running the Kickstarter to print the thing. I still haven't gotten all the shipping done. I'm done thinking about the book and I want to move on to drawing the next project, and my enthusiasm for doing all these other parts of the process is low.

Projects have ends and maybe this person's personal programming language has come to one.

I'm eighteen years into mine.

One possible step here is to step back from work, just look around at other things going on(either software related or other topics in life) and start learning some thing else.

For instance, one path look at courses/lecture-notes in pl theory/implentation, or some other aspects of software.

Just learn out of curiosity. This might lead to solution of original obstacle, or finding out that there are more interesting problems to work on. This happens frequently, even on success, some other project becomes more important than previous goal.

You have succeeded in your creation and are suffering because maintaining a product does not require the same set of skills. Maybe you should designate another leader for that. IMHO, creative mindset is not good for maintenance because you want to start too many changes at the same time. For maintenance, you need a more accounting mindset. Write how you plan the evolution of your product and gives this plan to someone else.

I think the ultimate problem here is lack of delivery. 6 years is a long time to work on something and not to get something done, delivered, completed and used. If the author wants to continue on the product id recommend going backwards a bit and removing all the excessive features from the product. Define the smallest set of features that makes sense and then complete those. Get a release done and then take it from there in smaller increments. I have worked on a personal small project (60-70kloc) for over 12 years and imho the key is t o be able to.deliver something . I nearly killed my project when I did the classic mistake of doing a complete rewrite which took me over 1.5 years. It was very slow labouros unthankful job with no positive feedback. In retrospect I should have just carefully refactor my old code base to reach the same end result but with smaller increments.

I don't think your point really applies here since as you can see on Github[1], the project has numerous releases, averaging at a release about every five months. Whether that's often or rarely is up for interpretation, but I'd hardly say the project is standing still.

[1] https://github.com/FascinatedBox/lily/releases

Oh, ok yeah. It just sounded like the had been grinding away for years without getting anything "out there".

You haven't provided anything that shows you've gotten any worse at programming. Just looks like you're fatigued and have accepted some of your own shortcomings that where always there but didn't have a stressful environment to manifest themselves in.

I try to be deeply convinced that my projects are not about progress, deadlines, or results. They are about working on them regardless of their level of completeness. It is very easy to think that the result matters, so I'm saying I try.

Ergo:

(1) If you care about the topic of your pet project, then rewrite it from scratch, avoiding all you feel is technical debt. It will take way less than six years to have the same features you have now (hopefully more maintainably then).

(2) If you do not care about the topic anymore, abandon the project. For a personal project, it is not lost work at all, because you've had your fun hours with it.

Tne author needs to read up on closure conversion [0]. What's described here (a transformation on the generated opcodes) sounds like a real mess -- hard to write, hard to maintain, and probably wrong in some cases. Time to be brutal, rip it out, and start over the right way. Forcing oneself to do that when necessary will definitely make one a better programmer.

[0] http://matt.might.net/articles/closure-conversion/

Programming can be art if you let it. At some point, damn the bugs, and work on what makes you happy. Start something new that excites you. Even if you never finish anything (or at least finish to your satisfaction if nothing can ever be truly finished), that's fine, artists that never go anywhere (and those that do) have tons of sketchbooks of half-baked ideas. A 6-year effort on a language is pretty impressive despite whatever monstrous bugs are inside.

But if you're not careful you'll end up gored by a bug and only be able to drink herb tea for the rest of your days. (http://www.workpump.com/bugcount/bugcount.html)

If you want to become better at programming, it's not fundamentally different than becoming better at other things. Take the game of Go for example since that's in my mind. Unless you're a natural genius, just playing a lot is no guarantee of getting good. You need to study. You probably need mentors and peers to play with and learn from. And of course you need to play for real. But always shoot for higher challenges. If you just play beginners all the time, you won't grow much, and may even regress by getting lazy.

I can feel the author's anxiery. I've created and maintained an open source project for almost 4 years and it's still not the most popular in its category though it's doing well and growing slowly and steadily.

I have at times thought that I must have done something wrong but those people who do use my project keep reminding me how useful it is for them.

I think it's easy as a project author to start blaming yourself for not making your project more popular but you have to keep in mind that project quality doesn't have that much to do with popularity. Some projects are just niche and you have to focus on making a small number of users very happy instead of making a large number of users a little bit happy.

Maybe your project could reach more people and the world would actually be a better place because of it but to convince people at such scales, it really takes a lot of marketing, connections and luck - This is beyond what you can do as a single open source developer.

I think that open source projects these days cannot get past a certain threshold of success until they raise funding and start seriously commercializing.

For what it's worth, the majority of my personal projects don't get far beyond concept.

Take a half full look: If you've created any software project on your own time that really does something that's great!

If you feel like you've made mistakes, learn from them. You may be learning more than if you hadn't made them.

Maybe redirect it? Let llvm / gcc handle the backend if you are sure the syntax is where it should be?

It will give you closure. The project will live on. What you wanted to accomplish, people writing in your own language, will live on?

It's all subjective. That you got bored out of it, is a sign of a healthy mind when faced with a repetetive task.

Best of luck with the next cool thing, whenever you get enough rest from this one ;)

Sunk cost, just stop

Sounds more like you've become more experienced and critical, shit happens :) Boring free-time projects will kill your soul; letting go takes practice, but just giving something away for free is plenty good enough.

Reminds me of "Worse Is Better" by Richard P. Gabriel: https://www.dreamsongs.com/WorseIsBetter.html

giving up on something is very hard. what helps is to think in micro economic conceps such as sunk cost and opportunity cost. if you could do anything more fun with your time you have to weight that in.

I see the author recently (1-month ago) re-implemented the Lilly language in C (It seems to have originally been written in Rust. In effect this is a completely new project based on learning from an older very similar project. Making a such a fundamental switch in the implementation technology is likely to have a negative impact on on bug count and stability that the author refers to. (Netscape went out business because they tried to re-write their browser in Java.) The solution here is for the author to realize that this is a new project and treat it as such and not worry about the state of bug count/stability too much. He probably just needs to soldier on to complete the project to their satisfaction.

> I see the author recently (1-month ago) re-implemented the Lilly language in C (It seems to have originally been written in Rust

That was an April Fools joke, Lily has always been written in C. :P

Ah. Thanks for the correction! This is the downside of a cursory examination.

That post is a joke (written April 1st).

Should have noted the date on the blog post!

I seem to be in the minority here (perhaps I'm the only one) but this piece struck me as a long humblebrag. "Golly gee, I'm developing this programming language and ..."

Also, new languages (kotlin, swift) and frameworks (vue, react) and practices (containerization, CI) keep coming out so you might feel like you're getting worse.

Honestly, trying to stay current on all the new technologies is causing me anxiety. After 20+ years at this, I always felt I was one step behind. But now I feel I'm 3 steps behind. I can't imagine what it's like to be a developer with a wife and kids. I just couldn't' stay on top of half what I'm able to do now.

Relax. I've seen code written with/for the newest technologies and it's mostly crap. Just like you don't have time to learn it, most people don't. But everybody is happy, including the client, because they have this new tool written in the newest technology so it must be good. People evaluate software just like they evaluate cars: they take a good look at the paint job and declare that this car is good, while in reality the seller only took care of the paint job, the engine is dead.

People often think new tech is a cure for their woes, such as poor code quality - often it is just papering over the real problems & barriers.

I've also seen tech switches used to mask rewrites, because they could sell management on the idea that $NEW_THING would solve our woes when what we really needed was just to spend half a year refactoring the legacy system (regardless of language), but half a year to redo work didn't fly.

Now that you mentioned it, I think this is the most common case.

Hi! waves

Developer with a wife and kids here,

You pick your battles and as you age you determine what's actually a priority to learn as well as developing a system to filter out what's not.

In terms of languages, I made the decision about 5-6 years ago that I was going to stop bothering to look at languages unless they solved a problem that I couldn't solve easily in another language. I tend to refer to these languages as "Flavors of C" since they all basically do the same thing.

At the time I decided that, I already had a good bit of experience with Java, Groovy, Perl, PHP and Ruby. As it turned out, it's been a good filter and the only two languages that I've actually felt the need to learn in the last 5-6 years have been Elixir and Go...and Go was largely because it's getting used at work. Elixir is the only one I've actually dedicated my spare time too and that's because once I dipped my toe in the water I got a little bit obsessed (because it's IMHO, the 42 of programming languages for my use cases).

You generally do the same thing with the devops and peripheral side of things too.

I used to manage my own servers in my spare time to learn, but I just don't have the time to spare for it. I stay on top of things related to my job at work, but in my spare time I just use Heroku because the $7 / month is worth it even though I'm sitting on a lot of digital ocean credit.

In terms of databases, after I went deep on Postgres with one project and realized that it basically addresses every problem I've ever encountered (personally) with a database that I just don't feel any need to learn another unless I get to an extreme niche scaling scenario where it actually might leave something to be desired.

I've entirely avoided the Javascript framework-of-the-week stuff. If I had to choose one to learn, just based purely on observation I'd probably go with React since it seems to be the most stable-ish environment but I'd be lying if I said I didn't see some real appeal from Elm too. You'll notice a pattern that when people keep trying to solve the same problem a different way, that there is probably an underlying issue as a contributing factor. Plus, 99% of web apps just don't need to be SPAs.

You just pick your battles. You stop learning technology in search of a problem and start focussing on technology that actually solves your problems.

Elixir-Phoenix-PostgreSQL is, in my opinion, the 99% solution and when you have a toolset that addresses so many concerns so well, the only time you'll find yourself looking for more solutions is in the gaps.

The fewer gaps, the less you'll feel the need to fill them.

Take a rest my friend. You don't need to stay current on all the new technologies, just to be aware of them. And most of those are basically easier-to-use or more-popular versions of things already existing in the past. The things that are truly new in our discipline, arrive slowly.

In my view there is almost no reason to stay on the Language/Framework Of The Month treadmill. If you were capable of learning language X, you're capable of learning language Y if you need to or just want to. Think for a moment: Why do you think you need to learn all the new technologies? Is it because you just love learning? If so, there's no reason for anxiety--learn at your own pace.

Is it so you can remain employable in tech? In my experience, good companies are looking for those "T-shaped" people: deep experience in one or two broadly defined domains, plus some experience across domains. You generally don't need to go deep in one particular language/framework combination. If you are good at, say, Javascript, and a company actually won't hire you just because you don't know React, well, trust me--you really don't want to work there. They are looking for a cog, not a creator.

Here's my totally anecdotal advice: Pick three or four resume keywords to go "deep knowledge" on in your career, and you'll probably be fine. Early in my career I happened to pick "cross platform" and "graphics SW". Mid way through I added "iOS" and "GPS/mapping SW". Well, I'm in my 40s and still going, so it turned out alright. Notice I did not specify any particular language or framework there: I have my favorites but I don't consider a particular language or a particular SDK to be that important in the long run. I have never really experienced this language anxiety. It doesn't matter.

Sooner or later you see the next wave and think "that doesn't seem worth the investment." And you skip that wave and the world doesn't end, and chances are the fad falls into irrelevance like 90% of other fads. And gets replaced by another fad which is basically the same with a bit of window dressing, and you realise the good concepts will stick around and the rest wasn't worth wasting your time on.

So you pick tools that work for you and are appropriate for the task at hand, and you get the job done, and you leave the fad chasing to the kids, because you've got better things to do.

> But now I feel I'm 3 steps behind.

Every time I see a new technology come and go, I feel myself was 3 steps ahead :).

Take a good look at any new stuff to determine what it is good for (if there's any). Keep your results in mind, and go back to your old stuff.

Then, if you have a problem that can be solved by the new stuff, go and learn it, otherwise why bother?

Honestly I don't see that huge influx of novelty people keep claiming is here. Yes, specialized knowledge is advancing rapidly, but mainstream tools are quite stagnant. (E.g. I'm avoiding learning how to script Docker for near a decade already.) It may be a matter of changing your focus from products into technologies (what does require deeply learning a few techs).

> (E.g. I'm avoiding learning how to script Docker for near a decade already.)

Quite an accomplishment, given that Docker is four years old. You're twice as effective in not using Docker as me.

Avoiding Docker as well like I avoided Ruby and mocking frameworks. Some things just don't feel right.

What? I remember reevaluating it so many times. Was there some other famous container product before it?

Well, four years is what Wikipedia states.

I don't think there was anything as prominent as Docker on cgroups that was as old as ten years, since cgroups are that old. LXC was started eight years ago, and then Docker hijacked the subsystem and nomenclature.

But maybe you were thinking about some predecessor, like Virtuozzo/OpenVZ or Linux VServer? Those were never as popular in general public as Docker, but they had their mindshare, especially in some circles.

Kotlin has been around over 5 years. If you catch a language early, you have plenty of time. If you catch it late, it can feel overwhelming.

I have to keep reminding myself that I don't need to jump onto every bandwagon. When I feel like some part of my workflow is too outdated or too broken, only then is it time to learn something new. I read about Vue and React, but I still use Backbone, jQuery and RequireJS because I know them well and can be productive with them, even though they're all (except maybe jQuery) considered somewhat outdated now.

I still use PHP too, although I follow it quite closely and use the latest language features, and it's not stagnant.

Welcome to the bottom of the curve! On the plus side, this means you're doing pretty well.

https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

https://www.psychologytoday.com/blog/head-in-the-cloud/20170...

Sounds like that the author needs to switch jobs or something. Try something out of your comfort zone. Maybe do something you feel is way harder but within reach?

>Solo developer projects are supposed to be awesome and consistent, right?

No. Where did the author get that idea?

I forgot my BASIC too.

The formatting of the text is awful on mobile. It looks like it's using something like <pre> to render to because there are hard coded new lines at the end of each line. Please don't do that, this isn't poetry! Just wrap your lines normally!

Reply on news.ycombinator.com