The idea you can't, or that you'll be less productive than an IDE user is just idiotic.
What has the language got to do with anything? Yes, java is sometimes verbose, so? If most of your time is spent typing, you're doing it wrong IMHO, not matter how verbose the language.
What magical feature of an IDE "allows you more freedom to explore ideas and experiment faster"?
Most people might write more lines of code or produce more classes in an IDE than text editor in Java, but most people aren't good programmers. Lines of code and number of classes often bear no relationship to productivity.
Essentially though, everyone uses an 'IDE', they just don't call it that... Mine is a text editor, bash shell and browser. All you need...
GUI design is a whole different kettle of fish. But a mockup builder is probably more useful for that.
Refactoring.
Code completion/validation/formatting (make fewer errors, and reformat external code to suit you).
GUI design (a lot of times you will have to make GUIs, no point in avoiding it).
Code analysis (on-the-fly code analysis and error-detection (syntax, compile, and run-time) = less creep-in errors, easier to spot errors earlier, and less compiles = more productivity).
Performance bottleneck detection.
Testing made easier (simpler an faster to build and do unit/integration/etc. tests)
Framework/Design-Pattern skeletons (got an idea, think pattern X may help, try it out quickly = make a good/bad decision faster).
Teamwork features (version control, changelist management, build management etc. made easier).
Etc. Etc.
Who said anything about more LOC being the/a measure of productivity?
An ex-coworker was arguing how great IDEs are, his comment being "The IDE makes it so you don't have to think" I think that's the best argument against IDEs I've heard so far.
If we ever meet in person, I'll buy you a beer/coffee/meal, and we can debate such things as in general I'm liking the comments you make on a multitude of topics.
An IDE is just a tool and like any good tool can help those that don't want to think, think less, and for those that do, think more?
That makes it more than a light switch, and will probably interfere with getting work done. Since, you know, many companies have standards about which languages and platforms they stick to.
Secondly, you are at liberty to write code in your free time, where you can experiment with whatever language/IDE/OS combinations you deem fit. I'd argue this will also make you a better programmer.
From a quick search on dice here are the number of job listings for lisp, cobol and java.
Lisp: 26 Cobol: 1007 Java: 15494