Your best case scenario is actually a worst case scenario.

From Düsseldorf to Frankfurt, using regional trains, you have to change once in Koblenz (RE 5 to RE 2). This is a connection which takes 3 hours and 50 minutes and runs every 2 hours (with numerous other options in between).