I glossed over it a bit in my first post, but the data on the drive is actually always encrypted. The system decrypts the data on the fly as it's read into memory, and encrypts when it writes to disk again. This has less of a performance impact than you'd think.
Also, just to be explicit on the user experience: when you boot the machine, one of the first things the kernel does is ask for a decryption password. If you cannot provide that, the system cannot boot further, because everything but the kernel and bootloader (in /boot) is behind the encryption.
1) So as described above, the storage is always secure, regardless of whether the machine is on or not. The rub is that when the machine is in use, the system is actively performing this decryption/encryption. I'm not an expert on the technical side of it (and it probably differs between implementations), but I'd imagine the OS keeps the decryption key in memory. This is functionally an instance of the "it's got to be decrypted sometime" problem, or a variation of the "analog hole" in DRM. At some point, the user will be accessing a decrypted version of the data, and if the attacker is able to take control at that point, he obtains the data.
Let's say I have an encrypted text file (on an unencrypted, regular system setup). To write to or read the file, I need to decrypt it. Maybe a program takes a password and opens a text editing window with the decrypted data. If I care about the security of that data, I'm not going to leave that window open unless I'm actively using it. I understand that if I leave that window open and someone walks by my machine, they'll be able to see the contents, because I left the data in an decrypted state. But I need to have that window open sometime, because I need access to the file myself. It's the same situation with full disk encryption--at some point the data will be decrypted for legitimate use (in FDE's case, only in memory, but still decrypted), and it's up to the user to protect it during those times.
If you lock your machine (using xscreensaver, slock, etc.) and set it to lock when waking from sleep (and whenever else), the attacker must circumvent the lock program before he can access the machine. Ideally this would not be possible (a lock program that lets someone access the system without the password is not a great lock), but there's always the possibility of some vulnerability.
Alternatively, you can always hibernate your machine instead of sleeping it. Hibernate writes the contents of memory to disk and shuts the machine off. When booted back up, the kernel finds the hibernated memory and seamlessly resumes from where you left off. If you set it up properly, the system will write the hibernated memory to an encrypted partition, so the session cannot be resumed without the encryption key. The downside is you have to type the encryption key every time you resume, and my encryption password is a good deal longer than my normal user account password.
In my own practice, if I'm leaving my laptop at a place I'm more worried about theft, I'll hibernate it. In normal use, I'll sleep it and rely on the lock program. Like I said in the earlier post, though, I'd love to hear if anyone has a better approach, or even an analysis of the security of some common lock programs.
2) I have no direct experience with SEDs, but I'm under the impression they decrypt the entire drive when powered on and the password is entered. Or else, they do the same on-the-fly operations I described above. As such, they would be vulnerable to the same attacks as above. Their advantage is transparency to the operating system and better performance. Also see Wikipedia's description of some of their vulnerabilities: https://en.wikipedia.org/wiki/Hardware-based_full_disk_encry...
3) FDE essentially answers the "how to efficiently enter credentials" question with "at the start, when you first mount the partition" ;). Aside from that, file-level encryption solutions definitely exist, and are commonly used. You can encrypt arbitrary files with the openssl command, and many sensitive files (such as SSH keys) have encryption built into their standard usage (SSH key passphrases). Even when running FDE, I keep a passphrase on my SSH keys, because there's nothing stopping a rogue program from grabbing them during normal computer use. I'd encourage any other extremely sensitive files to have their own protection. To quote tptacek, "FDE does basically one thing for you: it reassures you if your laptop is stolen from the back seat of your car or left in a cab." [1]. Other steps need to be taken to run a secure system.
However, I don't know of a file-level encryption solution that functions exactly as you describe. It would be tricky to implement, for the reasons you described, and others. For example, background programs write to the disk too, and sometimes what they write contains sensitive data. Are you going to enter the key periodically for their use too? And will all these programs play nice with the (comparatively) huge blocking times when writing while you type in the key?
4) Of course, that's the question with any security solution. Many of the Linux solutions are open source, so that's at least a plus, but certainly not any guarantee of security. Short of being a security professional capable of auditing the complete source, you have to rely on project reputation, recommendation, and (ideally) someone else's audit. I'm sure lots of people would like a better answer to this question!
Hope that all helps!
[1] https://news.ycombinator.com/item?id=9069669