This is why I find it impossible to work effectively in open-plan offices. I'm often looking off in to space, thinking. But I feel compelled to make clickety-clack noises on a keyboard, continuously and unproductively, lest people think I'm not working.
Easier said than done, but some unsolicited suggestions:
1) Acquire the self confidence that your results will speak for themselves, appearances be damned. 2) Realize your coworkers also value thinking, and recognize it properly. 3) Realize your coworkers are probably too busy worrying about their own appearances. 4) Any busybodies not too busy worrying about their own appearances can see through your unproductive busywork anyways ;) 5) Consider journaling / take stream-of-concious notes / mind-mapping. Might slow your thinking down some, but might compensate for memory some too.
I found myself unable to focus on thinking in the office, when there are other people present. I learned to compensate for that by doing my thinking in writing - I just start dumping my thoughts and talking to myself through my text editor. It feels a bit slower, but it does wonders for concentration and for short-term memory.
I like a text editor because I can type faster than I can write, but something about writing on a paper works better for me.
That said, I've recently started to go back to killing trees for things like daily checklists, as a physical reminder of stuff - and more "diagram" style doodling - all packaged into a three ring binder. At some point I plan to grab myself a document scanner so I can bulk archive my loose notes for convenience...
We also use slack and 'idle-ping' other team members first and keep conversations in slack if possible. This works quite well, too.
Recently we had an intern, which had a very annoying tic: While listening to headphones he now and then started to sing/whistle/hiss along, quietly but audible. It's strange and a personality thing, but it took quite a few days until I snapped and told him to stop. It also started again after a couple of days and I had to tell him again.
There a two things to note: For some people it takes a lot to complain about something and for some people it takes a lot to change a habit. In a large office this can be problematic.
I shall confess that I'm guilty of this, too. It helps, though, that everyone in our open-plan office has good headphones, so usually not too many people notice.
(And yes, I realize the bizarreness of this entire situation.)
The world does not need yet another million SLOC. We need to think a lot and type little.
Getting organizational buy in, though, was a lot harder. At review time, when we were measuring employees against each other (don't get me started on the stupidity of stack ranking, but that was the system we had to follow), other managers would try to use it against my team saying that they're always away from they're desks and trying to compare them against their own team's long hours and heroic firefighting of production issues. It took me about 18 months of convincing before I got our directors to realize that my team was producing just as much functionality, with much more complexity, and doing it while seeing a fraction of the production issues that other teams saw. I had to get testimonials from our support team along the lines of, "we don't answer many questions about their stuff, it just works."
By the time I'd left, I'd just started to get through that busy isn't a virtue and hard work is only a good thing if it can't be avoided. Time spent reflecting is crucial to doing the least work necessary. Skipping that time will only create the appearance of productivity.
That would be a solution to a non-problem ("looking busy").
I use Emacs for that, but because I use Emacs for almost anything and have a rather low font size set, cow-orkers doing casual screen glancing can't even tell if I'm coding, working through TODOs, requirement docs, or if I'm IRCing or reading my mail.
That, and the fact that open-plan offices are, in my experience, a noise hell. There's always seperate 2-3 conversations happening in the room, some of them about actual work, but disturbing 20 people nonetheless.
Somehow I don't feel the same rules are going to get applied to the rest of us.