Stepping Back
rjp.io
rjp.io
I have a churn of tasks and subtasks that I add do, my brain randomly picks one to be the most important thing in the world and I do that to the exclusion of all else.
Methylphenidate didn't help much but dexamphetamine seems to improve things a bit once I get past the drowsiness it causes.
Sometimes it does it right, some times not. I can see the relation to gambling if, say, it does it right 50% of the time. If I had been taking a more scientific approach to the problem and had a clear direction of what I wanted to test, I suspect I wouldn't have gotten quite as "stuck".
That's a better description than what I came up with, "Tiktok for engineers". LLMs probably compound the issue, with that hope of a magic outcome. Though I've had many problems pre-LLM where I was plowing through it myself and couldn't let up...
Twenty out of a hundred gens are malformed horrors. Seventy out of a hundred are boring or cack-handed. Nine are interesting and worth exploring. And one out of a hundred I would say the algorithm accidentally produces real art.
Roll that one out of a hundred too many times when you're just starting out, and, well, you end up like me.
Two things that should be available by now in conversations are branching and history editing. Branching is somewhat trivial so let's focus on the history.
Now when I last used an LLM API, the whole context was still managed manually. Meaning I had to send the API the whole conversation history as one long text for every new query.
This means that I could technically change a part of the history directly. Manipulating the history though is not really a trivial problem. The LLM would need to re evaluate starting from that point.
But, the re-evaluation may result in something completely different... If there are branches, perhaps it would also be desirable to let it propagate into the branches.
Next, re-evaluate until where? We can assume a conversation happened until the present moment and the user may have changed their reality/state during the conversation before that point. For example, I may have changed some function based on a suggestion of the LLM. Now, for re-evaluation it would actually be nice if the LLM could also take that state change into consideration.
Here it would be nice if the LLM had the concept of certain logical facts and pieces of information and how they relate to each other but with an interface so that we could see that. If such a piece of information in the conversation is then changed, that would affect the information that is related to it. We could follow a sort of sequence of logical conclusions being made to verify what happened.
Just some thoughts with no conclusion. I think current LLM interfaces could be a lot better.
Let's say you ask an LLM to help you with a programming task. The LLM gives you a solution and you copy that into your code. While getting to the result of this solution, spanned over multiple messages, you give it more context about your code base. You ask it to help with multiple sub problems and it answers with suitable responses containing code snippets.
Now, you change your mind and want to actually write your program in a different language. You could at this point just tell that to the LLM and it would change course from here on out. Everytime it is queried the full history including the original problem is sent to it. The change in programming language is taken into account because it eventually gets to the point where you changed your mind.
But instead what you'd probably want is the solution it gave you initially and all the little solutions to all the sub problems but in the newly chosen programming language! You'd like to simulate or re-evaluate the conversation but with the newly chosen programming language.
I haven't seen that implemented in a usable way but as I said, this is not a trivial feature either.
You can't delete messages, you can't branch, and you can't edit the LLM's messages. These are basic, obvious, day-1 features I implemented in my own LLM UI that I built over 3 years ago, yet I'm still waiting for flagship providers to implement them.
It's pathetic how rare good UX is in software.
Can you list some UIs with these features?
In a way it is "re-evaluating" the entire history for every token it generates.
So yes, I know that I can just do a simple text replacement somewhere in the history but that's not really useful. I want the conversation to be re-evaluated from that point because it might have gone differently with that changed message.
The way this is done, technically of course, is you build the "Context" by walking back up the tree parent by parent, until you reach the root, building up 'context' as you go along, in reverse order. That will then always be the 'right' context, based on branching.
Related to this I've learned that a lot of the time it's best to just use a "Context File" which is a big file of instructions explaining goals, architecture, and constraints so I can just tell the AI to read it and then say "Do Step 21 Now" for example, if step 21 is laid out in the file. This way I sort of can micromanage the context, rather than trying to reuse old discussions which are usually 90% throw away content, no longer appliable.
Prapti can do editable history no problem, the entire history is just the markdown file. We experimented with branching using git to automatically switch branches based on how you edit the history.
I still use it frequently. I prefer keeping my chats in local markdown files.
It’s really unhealthy behavior.
But you really need to step back once in a while and contemplate, if the thing you're doing is really worthwhile.
One of the most precious resources is time. I didn't appreciate this insight a lot while I was young. But as I grew older I needed to be more careful how you I was spending it. In this regard like I like the saying "youth is wasted on the young". But this also enables you to be more focused in your approach. Fail fast is a lot better than spending years on a problem with bo end in sight
I also don’t think it’s possible to focus at 100% on a detailed complex problem, and also concurrently question is there a better path or a way to avoid the current problem. Sometimes you just need to switch modes between focusing on the details the weeds, and popping back up to asking does this even have to be completed at all?
But it’s so scary to depend on that flash of insight, after all it’s not guaranteed to happen. So you keep grinding in an unenlightened state.
If there was a Reliable way to task your subconscious with working on a problem in the background I could probably do my job in a third of the time.
So for a software project, I break it into parts recursively. Sometimes for a module, I can build some scaffolding even when I'm not sure about the full implementation. I can go up and down that tree of problems and I always have something to break away from my current thinking track. So there's always a lot of placeholder code, stubs, and various TODO comments around my codebase. And I rely heavily on refactoring (which is why I learned Vim, then Emacs as they helped me with the kind of quick navigation I need for that approach). Also REPL and/or some test frameworks.
However I’ve noticed that too much refactoring while exploring the problem space becomes tedious for me. At that point I go back to a notepad or whiteboard. Maybe this change of medium is another way to engage the subconscious?
And to your point about vim/emacs - the choice of tools no doubt impacts your engagement with the problem at hand. That’s why having AI generate teams of code doesn’t work for me - but using it to iteratively develop some PoC code whose core I then incorporate into my own code works brilliantly.
Yes, I'm only writing code when I got a good idea on what I want to write. Most of the times, I'm doing research or doodling design ideas. Refactoring is when I've thought of a better approach for something. It's almost always easy to do as big refactoring only happens at the beginning of the project, when not a lot have been written.
> That’s why having AI generate teams of code doesn’t work for me
It also doesn't work for me. Because it's always too much implementation details at the beginning of the project. I'm keeping things loose on purpose while I'm exploring the domain. And when the times comes to finalize, it's mostly copy and paste as I have all the snippets I need. It can serve as good examples though for libraries and frameworks you don't know much about.
In my hobbies I think going off on tangents _is_ the experience, so I have no qualms about doing it. But when I’m working I’m almost always thinking in terms of stewardship. Is what I’m doing good for the business, our customers, my coworkers, my own professional development, etc.? Any sort of fixation on minutiae is just subservient to those questions.
That said, I don’t think of a 5 minute tangent like the author describes as a heavy investment. If anything it’s just a happy little side-journey taken to better understand the nature of the problem space. For me the threshold of “this may be a waste of time” is more measured in hours than minutes.
I am not sure how much I believe that or how true it is, but I have found that many times I have come up with a better solution to a problem after going for a run or having a shower. So there might be some truth in it.
But yeah it is hard to know when you are in too deep sometimes. I find that imposter syndrome usually kicks in with thoughts of "why am I finding this so complex or hard? I bet colleague would have solved this with a simple fix or a concise one-liner! There must be a better way?". TBH this is where I find LLMs most useful right now, to think about different approaches or point-out all the places where code will need to change if I make a change and if there is a less-destructive/more-concise way of doing things that I hadn't thought of.
It's something I've actively used for almost two decades now when dealing with challenges i'm stuck on. I remember one of my professors explaining it as having a 'prepared mind'.
What I do is, before I go to bed, try to summarize the problem to myself as concise as possible (like rubber ducking) and then go to sleep. Very often the next morning I wake up with a new insight or new approach that solves the problem in 10 minutes that took me hours the day before.
Something that gets in the way of people’s understanding of themselves (not accusing you of this) is thinking that they’re aware of everything going on inside their brain. This is obviously not true.
You can’t catch a ball by being aware of the angle of every joint as you do it. You can understand someone speaking by considering all the rules of grammar and vocabulary as you listen. It’s just too slow.
It’s like a CLI program in verbose mode. Even with stuff flying across the screen, you can’t print out everything that’s happening. It’s just too much.
While you sleep, your brain is rearranging itself to solve your problems. It would be just as accurate to say you are rearranging your brain, because your brain is a part of you and you are a part of it. If I give someone a handshake, my hand is touching their hand and I am touching their hand.
Keeping the conscious mind updated on this whole process at all times would be like telling a PM about every keystroke.
I find this works extremely well.
It can really be a tactic to overcome some things that probably could not be solved any other way.
That's the kind of rest I like to get, where you make progress at the same time ;)
To contrast with for example your biceps, which will use a lot of energy when lifting something and then scale back down to near-zero.
So in terms of energy use, yes, the brain is always going at near full burn.
On taking a break, I've often found it difficult to stick to the pomodoro technique because the breaks seem too frequent. But those breaks help avoid going down an unfruitful path for too long.
Then concept is still the same, in terms of focusing on the topic at hand. I no longer count down to my breaks, I just count up until I want/need to take a break.
It's basically just a glorified time tracker, I guess, but it was my solution to the pomodoro "break" problem, i.e. sometimes I don't want to take a break.
I embrace the subconscious basically.
One upshot of this is that it forces me to be clear about the definitions of "going to work" and "not going to work." It's related to definition-of-done in some agile methods, and also to "imagine the problem as solved and work back from there," which is one of the heuristics Polya talks about in "How to Solve It".
Personally I've found a continuous open awareness style of meditation has really helped me balance things out, as I went from someone with very little doggedness to e.g. being two weeks into cataloguing all my books with Delicious Library before realising it was kind of pointless. The open awareness practice (very different from focus-on-your-breath and also the visual deconcentration discussed in the link above) is about encouraging the recognition of this-is-how-things-are as it naturally and spontaneously occurs; doing this more and more also builds confidence in one's intuition about (in this context) whether to persevere at the current task or whether to step back.
I don't think there is a foolproof system that can be developed as a substitute for this kind of intuition. Speaking for myself I can and will continue to second-guess over time any external system that may feel definitive when it is first established. But I can learn to trust the (unfortunately) indeterminate part of myself that tells me I'm doing the right thing.
The main Hamming materials I'm aware of are:
"You and your research" https://www.youtube.com/watch?v=a1zDuOPkMSw
and the course, "Learning to Learn" https://www.youtube.com/playlist?list=PL2FF649D0C4407B30
The original course notes are online somewhere, can't find right now.
And I just found that there is a book: https://gwern.net/doc/science/1997-hamming-theartofdoingscie...
A serious talk by British comedian John Cleese "Creativity In Management" is also relevant https://www.youtube.com/watch?v=Pb5oIIPO62g it touches on switching between "open" and "closed" modes of thought. Perhaps this is what we'd call switching between the DMN and TPN. Neurotypical exclusive switching between these networks is a function that is theorised to be impaired in ADHD.
You can think of it as a RL problem, and there are some interesting algorithms which achieve good performance by periodically 'breaking out' of exploitation, but less and less: https://arxiv.org/abs/1711.07979 (I expect that you can come up with an infinite hierarchy of 'wake ups' which converge to a fixed overhead and which aren't terribly far off an optimal schedule of replanning wake-ups, and that something like 'great thoughts Friday' is the first step; then you'd have 'great thoughts first-of-the-month' and 'great thoughts new year's day' etc: https://gwern.net/socks#fn10 )
Generally I think it is a question of method. If I was to produce a physical thing I would know the steps. One of steps would be a drawing in a known form, with known ways to indicate what is important. With software there is no such a standard form.
The key is to care deeply about solving the problem, not simply looking to get rid of it for a quick win, if you are immediately thinking about something else as soon as you can, you'll never get the best results.
When using code assistants I found that a nice way to take a step back is to use the Socratic method, chatting with it about the design instead of trying to force it doing it for you. It is like chatting with a well informed but not too clever normie colleague, it can help taking a step back.
What I tell myself (which is true) when I notice I've perhaps done something where the effort might not be worth the pay off, is that life itself is about the journey not the destination. As they say "The Journey _is_ the Destination", which applies to my other hobby, riding motorcycles. You can always get anywhere "better" in a car, but it's more fun on two wheels.
I often talk about how I think people should learn to code not to get a job but for the mental models it gives you.
Similar to how learning a musical instrument enriches your life.
Or art.
And one example that feels particularly apt here is that in my art classes in college, my drawing professors really hammered-in the habit of stepping back a few feet from your canvas.
It might feel like it's applicable specifically to that context, but it's amazing how much that voice comes back even when I'm working on code or 3D modeling: "take a step back."