"people had fewer than 3-4 hours per day to actually work" had the same experience. the unwritten expectation was to pull 8 hour coding days /plus/ all the Scrum overhead. if you don't like it, there's 1000 guys waiting to take your place, partly thanks to remote. a core reason we haven't fire you yet is that it is costly to ramp up new people, but don't try our patience.
see also "hire fast fire faster"