A 4 hour coding assignment is half a day of work. I don't care if it is a toy project and if there is no business value in the work. But a 4 hour coding assignment is still half a day.
I may have to take a leave from my current work, take half a day off off my current contract or spend a significant part of my weekend on this problem, instead of with my family. I certainly expect to be paid for it.