How to control the metacognition process of programming?

Looks like we're hugging the author's site, so here's this: https://web.archive.org/web/20220409185007/https://lambdaisl...

I've been working for years on a good set metacognitive debugging questions. The most useful ones typically have to do with physical and emotional self-awareness and task prioritization.

I notice I'll get stuck in all kinds of mental loops which are not particularly useful, but always indicative of something I should consciously review. I actually have a personal Slack workspace I use as a hybrid between a journal/log and a conversation with myself. This technique has helped me achieve record levels of consistency, productivity and motivation.

From the article:

"Sometimes, when I want to find someone to discuss, I need to wait. Several times, when I encounter a bug and I want to ask for help, I write down all the details about the code, the environment, and how I have attempted to solve the bug. After I paste my bug report in the Gaiwan’s communication channel and take a 15 minutes break, the magical thing just happens: the muse comes to my mind. I solve it quickly and edit the message that I have already posted. The Rubber Duck Method really works!"

+1 to that!

Some of it is literally beyond me. I am pretty sure my subconscious has a good thirty IQ points on me. I am a decent enough programmmer, mostly through habits and attitude, but sometimes I will have to just ... fiddle around, do the boilerplate, write documentation, whatever, until my subconscious throws me a bone and I get the algo I need. This often happens while I am falling asleep or waking up, whole diagrams just appearing in front of my mind's eye, everything explained.

This sounds a little superstitious but my intuition rarely leads me astray. It warns me, "Don't take that job, don't trust that person, you should go over here" very subtly and I have learned that if I ignore it, I will usually regret it. The job I felt I shouldn't take, despite it sounding perfecto, got hit with huge violations for OSHA and other things. The person I distrusted on instinct is later found out to be someone I would rather not be entangled with, and so on.

I don't know what that guy in there is doing.

I now baptize the "ask a rubber duck pre-made questions": the Rubber Drucker technique.

This is an interesting post. There is much discussion about software and whatnot here but not about the stuff of which software is made: thoughts.

The way I handle thinking while programming is to think through writing. I handwrite much of my thought process while programming. Which allows me to stop at any distraction and come back to work easily. I call my writing: my secondary memory; and my working memory: my RAM.

I hope to see more posts of this kind by a diversity of people. "What do you ask yourself or do when [insert situation here]". About programming and more.

I often struggle with the article's described problem when programming. Bug in own added code leads to stress, hence debugging in stress mode. I do not feel it has much to do with muscle memory while typing, more with the effect of anticipating/hoping for the best outcome while testing. What helps for me is to change code (tiny refactorings or adding functionality) and test in baby steps when I'm not very confident. This would filter out any wrong assumptions of existing code or misunderstanding of the system on-the-go without much stress. A big disadvantage of this approach is that it can be tedious and time consuming. Other approaches are to make yed drawings and like the article proposes, rubber ducking to a real person or by writing down the description of the problem and related questions. I secretly hope to get better or faster approaches in further comments,to reduce stress while debugging.

It's an interesting question, designing a system involves balancing constrains and reducing coupling to be able to iterate and adapt at low cost. Designing a single function is also balancing constraints. type safety, edition cost, boolean space reduction, gotta get a sense of what basic function to write to then grow it and converge fast to a fully working one (reminds me of HtDP regular function design process actually)

This article helped me to realize just how useful meditation and spirituality has been to my programming workflow. I'll start with meditation, and build from that to spirituality so as to reduce the level of woo, I know how much HN hates woo.

A long time ago meditation brought awareness to a process of 'clearing your mind'. Our minds are constantly under bombardment from the senses, meditation creates a state free from bombardment, however just being free of external influence doesn't mean you can actually take advantage of the freedom in a useful way. This, naturally, yields to practice under instruction, which I was lucky enough to obtain. At one point I thought to take my practice outside, moving meditation would up being key to unlocking a permanently clear mind.

Many years later I came across the teachings of George Gurdjieff, whose main technique of 'remembering' would not have been possible without the long hours and clear mind obtained from meditating. Remembering is a low-grade meditation where the object of meditation is the amazing nature of our selves. Usually when meditating, you hold an object in mind for the duration of the session. Here, you keep an object in mind constantly. Forgetting, as you imagine, is inevitable. As soon as you forget, a period of time passes until you remember again, that you knew this thing about yourself.

This episode took a few weeks, but in the end I obtained the ability to not just remember myself, but also any concept I cared to work on. Key to the process was the intuitive realization that my mind had a 'front burner' and a 'back burner'. You can only have one thing on the front burner, and meditation is useful for making sure what's on the front burner, stays on the front burner for as long as I want it to stay on there. Seated meditation keeps it there for short periods of time, walking meditation keeps it there for longer periods of time. Remembering meditation makes it possible to keep something in mind indefinitely. The back burner is a function of the subconscious that I never knew about or could harness until I experimented with the teaching.

At the time I was doing the 'remembering' exercise I was working at a small IT outsourcing firm. Since these meditation exercises take weeks to conclude, a good portion of this happened at work. I remember being somewhat scared that all this meditation would make it impossible to have and keep a career where I use my mind constantly. At first, just trying to do any work at all would make me forget. Eventually, I could take my active attention off of what I was trying to remember and in the back of my mind I could keep what I was trying to remember. This led to the 'back burner' terminology.

When I'd take what's on the back burner and put it back on the front burner, I'd notice it changed. Advanced. Evolved. What this means is that while I wasn't thinking about something, my mind was still operating on it. At the time I didn't realize how amazing it could be. I thought I'd just verified that we do in fact have a subconscious and it does, indeed not rely on the conscious mind to process.

Here's where it starts getting spiritual. Woo incoming.

Around the time I first started meditating I was also taught the foundational basis of spiritual practice, which is, belief can be worked on. You can go from a state of not believing in anything non-material to a place where you fully believe in and exist in a world dominated by spiritual rules rather than physical ones. I did not complete this journey at the time I was taught it but rather much, much later. You just 'act as if' it were true and eventually your brain comes around to believing in it. Surprisingly, you can explore an amazing quantity of very woo-filled paradigms without actually believing in any of it. (you basically adopt a research mindset) But when you do finally believe in it, it flips a switch in what's possible. (here's the woo if you were looking for it. I can explain but we're talking massive length that most readers just aren't in for. I have a Quora account you can comb through if you're really interested. https://www.quora.com/profile/Vincent-Guidry-1) Chakras, Intention-Manifestation, I've explored it all, pre- and post-belief. I won't go into event that flipped the switch because it would be at least as long as this has been so far. But the important part is that I did eventually start believing in spiritual existence some 7 years ago.

To take this back to topic, this would became important later on in my programming career. I believed in this amazing existence, but how does it apply to my professional life? My spiritual awakening actually coincided with my entry into the software field. Quickly I would run into the dilemma, what takes precedence, my ongoing meditative and spiritual explorations, or my worldly money-making life? Because I believed in it, I was comfortable taking leaps of faith. I was just going to do what I wanted, and if I got fired, well, I got fired. Just get another job, or whatever, mine is a present-focused life, I can sit on park benches for hours and be happy, unemployment is nothing to be scared of.

But the funny thing was, I didn't really have to sacrifice anything to have both a spiritually-driven life and also a financially-abundant career. I could simply rely on the older meditative advances, keeping my work stuff on the back burner, and if that failed me, well, my spiritual belief would kick in and make me totally comfortable with whatever happened as a result.

But well, I always found that the subconscious would deliver insight that I didn't have before, if I left it alone long enough. At first I was concerned with questions about yaknow, how long does it take, exactly what constitutes taking my mind off of it, because I actually care about my career at this point, whereas before I simply didn't, meditation / spirituality took first and only priority. I'm not actually looking to get fired.

But over time I realized that most coders take about the same time as I did to work through problems. I just didn't spend all the metacognitive time being anxious over it. The spiritual stuff was what gave me the freedom to stop caring about it, and the meditative stuff was what gave me the actual skills to just totally move my brain off of an active concern and trust that my brain wouldn't just stop working on it.

And a lot of the content of the eureka moments I got after putting the current problem on the back burner was to realize that a great deal of these problems weren't going to get resolved with code. That I needed to be raising issues to my boss or even higher. This raised my profile in whatever company I was working for. I wasn't just solving code issues with my purposed 'not dealing with the problem so I can let my subconscious deal with them' episodes, I was solving organizational issues and in a small number of cases business problems as well.

Because, and I realized this much later, I'm not just not dealing with problems when I put them on the back burner. I'm reading reddit articles, Hacker News, deep spiritual stuff, whatever it was my 'soul' wanted to explore. My forebrain is working on integrating the combined knowledge of the internet and the world into my subconscious understanding. So when I'd finally 'remember', that is, bring what was in the back burner back to the front, it was someone who had just gone through a soul journey that was dealing with the problem now. It made my problem solving mind better.

This turned out longer than the blog post inspiring it but if you're still reading and wondering, 'how can I?????', my best advice is to start with spirituality, not with meditation. It's faster. You have to put up with woo, but if you can get past it, your life will never be the same. It's not hard to find my email / Quora profile, my DMs are open. http://www.integrateddaniel.info is the best starting resource if you must begin with meditation, but you'll need instruction to turn that into spiritual faith. Hard to connect it all.

I always tell people most programming happens away from the computer.

Ask yourself a question. Instead of trying to find the answer with you conscious mind, leave it open. Some time later the answer comes up from your subconcious mind.

Lambda is not land.