I've heard a lot of claims about lucid dreaming among some young people I know well locally (some of whom post here from time to time). I don't get the impression that lucid dreaming is really as beneficial as they think it is. Some cases I know about from personal observation involve disturbing sleep cycles so much in pursuit of lucid dreams that the young people failed in work environments or crashed and burned in their university studies. Getting a normal amount of sleep (for you, that leaves you feeling rested when you wake up in the morning) is very important. It's a lot more important than what kind of dreams you have.

I read through the whole submitted article, and I didn't see any reporting on rigorous study of the waking state health or performance of long-term lucid dreamers. All we see in the article is anecdotes, summed up by

"For Hobson, the neuroscientist, the benefits of being able to achieve lucid dreaming are much simpler.

"'We don't really know if there are real psychological advantages, but I can tell you that it has huge entertainment value. It's like going to the movies and not paying for your ticket.'"

An issue to consider whenever participants on Hacker News discuss self-help strategies is how reliable the research base is. People who only use the University of Google Library to do research will often find websites by advocacy groups that are pushing a solution that may not have been tested. Fortunately, Google's own director of research, LISP hacker Peter Norvig, has written a guide to reading research reports

http://norvig.com/experiment-design.html

that reminds us all about what to look for when someone reports some new, amazing treatment. Check out whether lucid dreaming has really been well evaluated with sufficiently large sample sizes, control groups, and other marks of good research.

I think a writing intervention

http://homepage.psy.utexas.edu/homepage/Faculty/Pennebaker/H...

may do more for many people in high-creativity careers than lucid dreaming. There is a better research base, by far, for the writing self-help than for lucid dreaming. Try it and see how it works.

I've always been able to enjoy interesting movie-like experiences by DAY-dreaming, and it's not explained here why anyone should alter their sleep cycle (which has known risks, up to and including psychotic symptoms) just to be entertained. A lot of researchers over the years have done a lot of research on human dreams in particular and sleep cycles more generally. Where is there any evidence that lucid dreaming is helpful rather than harmful, long-term?

Best wishes for much success in improving your personal insight and problem-solving.