"Learning" here means averaging over many rollouts (i.e. simulation runs) of a policy. Naturally, time only increases in those runs, as you require.