Particularly for Java, where there's a lot of boilerplate that can be autogenerated IDEs are very useful. IDEs are great for method signatures and completion, as well as integrated debugging and unit tests (1 click run the tests again, add in your breakpoint and check your variable).

Some languages lend themselves to IDEs easier than others. For Ruby I use textmate, for Java I use eclipse. For Perl I use vi, and when I did .NET I used Visual Studios.

IMO, the little things add up fast. Syntax highlighting can make things easier to read. Linking ('s and )'s let's you close a conditional without counting the number ('s less the number of )'s. Syntax aware re-factoring let's you safely change all x's to index etc. Built in language support allows you to catch many syntax errors before you hit compile and the help goes on and on.

PS: And don't forget the big things like built in debugger support.

The main things I like in an IDE are discoverable functionality (e.g. in Visual Studio, variables that changed since the last breakpoint are red--I'm sure gdb has acquired this ability since I first looked at it but it's questionable if I'd've ever gone looking for it on my own) and refactoring (e.g. changing the name/signature/return type of a method throughout your codebase). IDEs are typically big wins when you're writing in statically typed languages (especially ones without a REPL) and less so in dynamic ones - in Python I'd be perfectly happy in Notepad for at least 75% of my programming because I do so much exploratory programming in the REPL.

Easier debugging, faster code navigation, better refactoring tools, auto complete, inline documentation, tighter integration of unit tests, and probably many other features I'm not thinking of. Sure I write one off python/ruby/ps scripts in notepad++, but for me any project of real size and complexity will receive a huge productivity boost from a well configured IDE.

Your mistake here is thinking that IDLE is "the" IDE for python. It is really nothing like most real IDEs, and more of a toy.

As some others said, the language makes a big difference. When I write Ruby/Rails code on my Mac, TextMate and a terminal window are all I need. When I'm at my day job writing C# (a static typed + compiled language), a setup like this would be incredibly painful. Just the loss of the powerful Visual Studio debugger would send me home crying, not to mention having to worry about setting up complex dependancies in a build script when calling the compiler over the command line.