A perennial topic of discussion among creative people.
Your issue with distractions is almost certainly a negative. Sometimes you've simply got to face those head-on. It's very, very difficult to handle both these and creative work. A sense of fatality about whatever's causing the distractions may help -- there's something to be said about Ashleigh Brilliant's "I feel much better now that I've given up hope".
Csikszentmihalyi's book Flow is probably the canonical reference. Read it. He's also appeared in multiple interviews (I encountered him a couple of decades back during a long road trip through the back country, it was a wonderfully serendipitous discovery).
I also listen to numerous interviews with authors, musicians, artists, and similar creative types, including occasionally scientific researchers. I'm finding these to be increasingly interesting with time. The task of getting yourself into a state to allow creativity or problem-solving to emerge is one shared with many others.
As for my own observations:
• Routine helps. Having a regular "beat" to my day provides structure.
• Recognizing my own limits and rhythms is key. As I posted elsewhere in this thread, there are many forms of physical exercise and fitness practices. For many of them, a relatively brief training period (as few as 4 minutes, generally up to an hour or two, rarely more) provides optimal benefits. Rest and recovery are esssential. I'm finding similar patterns in mental tasks: there's only so long I can remain focused.
• Removing distractions and concerns helps greatly. At their best, academic environments are almots coddling in their embrace, and allow freedom from immediate concerns to explore and analyze topics in depth. Many corporate or government research environments similarly remove these concerns. Frank Herbert's Dune includes the litany against fear: "I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain." I see stress as the mind-killer, and its negative influence on creativity and flow is legion. Mind: short-term pressure can lead to focus, but chronic stress is an absolute disaster.
• Engagement with the problem at hand and belief in the mission helps greatly. Among my frustrations with present tech trends is that much of the work is what David Graeber calls bullshit jobs: http://www.strikemag.org/bullshit-jobs/ I've expressed those frustrations here on HN more than once, and have been working for the past two years on my own response to this. A key problem is that non-bullshit jobs and compensation tend to be inversely correlated. Working for something I don't believe in, however, is ultimately as destructive as chronic stress (and is likely a form of it).
• Creating an environment in which you can work without distractions and which is filled with calming influences helps greatly. I'm fortunate to have a space of my own, a physically comforatale environment (not too hot, not too cold, good lighting, excellent seating), free from interruptions. I've found myself listening almost exclusively to classical music, and avoiding (at least for much of the day) news programming. I avoid virtually all contact with advertising, which I find has a tremendously negative influence.
John Cleese's presentation creativity is another I've found insightful. In particular, he points to the role of humor in creativity: it's the brain's response to surprise (or one of them), and may exist in part to help us be aware of a novel circumstance or phenomenon, one which might prove useful to us. It drops our mental defenses against novelty.
http://fixyt.com/watch?v=AU5x1Ea7NjQ
I won't claim to have licked the problem -- it's a process, not a product. But I've made a great deal of progress in exploring ideas and issues that I'd been wanting to look at for years or decades. And it absolutely makes me all but positive that many trends in business and workplace practices are very highly counterproductive for creatives: open-plan offices, communications overload, business-faddism in general, brainstorming, constant progress reports and evaluation, punitive consequences, lack of opportunity for professional development, treatment of employees as disposable, and badly broken evaluation metrics (themselves a characteristic of trying to assess quality of a complex domain, see Pirsig's Zen and the Art of Motorcycle Maintenance, which I've been revisiting.