I'm not sure that is true.
Firstly, what are you working in exchange for? Are you performing some kind of useful work for someone else, receiving nothing in exchange then going and being a hunter gatherer for some reason? Probably not.
>You didn't work for a few hours then order food for dinner
Money has existed for a very long time so purchasing food ready to eat isn't so far fetched assuming your life took you near a non-trivial population center. Bartering goods like food, clothing etc has existed for even longer.
>You probably worked a few hours, then spent the next many hours either harvesting and/or creating all the requirements for life. Maybe you spent 2 hours hunting a deer, then an hour creating clothing, then an hour cooking, then an hour mending the hole in your roof, etc.
It reads as if you are assuming that a given individual is going to have to do everything, which doesn't seem to have been the norm for most humans. Most people didn't live alone. Instead of doing all of that maybe you spent 2 hours hunting a deer. Meanwhile someone else created clothing, someone else cooked, someone else mended the roof.
Its also worth noting that all of these jobs have a very well defined point of being done. It doesn't matter if it takes you 2 hours or 20 minutes to acquire food. Once you have as much as can reasonably be consumed before it goes bad you are done. If you catch a deer within 30 minutes you are done. If someone else did well fishing then there is no value in going hunting at all.